2 Repos
Awesome GitHub RepositoriesInference Optimizers
Tools and techniques specifically designed to increase the throughput and reduce latency of AI model predictions.
Distinct from Performance Optimization Utilities: Distinct from Performance Optimization Utilities: focuses on ML-specific techniques like dynamic batching and multi-stage pipeline orchestration.
Explore 2 awesome GitHub repositories matching devops & infrastructure · Inference Optimizers. Refine with filters or upvote what's useful.
Awesome Inference Optimizers GitHub Repositories
Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
- bentoml/bentoml
bentoml/BentoML
8,456Auf GitHub ansehenBentoML is a machine learning model serving framework and GPU-accelerated inference server designed to package, deploy, and scale AI models as production-ready REST APIs. It functions as an AI model lifecycle manager and an inference graph orchestrator, enabling the chaining of multiple models and custom logic into complex pipelines for advanced task sequences. The framework distinguishes itself through a dynamic batching engine that optimizes GPU throughput and an artifact-based packaging system that bundles model weights and dependencies into immutable archives for consistent deployment. It
Optimizes CPU and GPU utilization through dynamic batching and pipeline orchestration to speed up predictions.
Pythonai-inferencedeep-learninggenerative-ai - vladmandic/human
vladmandic/human
2,999Auf GitHub ansehenHuman is a TensorFlow.js computer vision library used for face, body, and hand tracking within the browser or Node.js. It provides a framework for human pose and gesture tracking, facial recognition, and biometric liveness detection to verify a live human presence. The project distinguishes itself through a full suite of identity and motion tools, including a facial recognition framework that generates embeddings for similarity matching and a background segmenter for separating humans from their environment. It incorporates a liveness detector to prevent spoofing during facial analysis. The
Optimizes inference speed by selecting the fastest available hardware backend and implementing model caching.
HTMLage-estimationbody-segmentationbody-tracking
