1 repository
Awesome GitHub RepositoriesInference Stream Parallelism
Techniques for distributing inference tasks across multiple hardware contexts and non-blocking streams to maximize throughput.
Distinct from Tensor-Parallel Inference Distributions: Distinct from tensor parallelism as it focuses on execution stream concurrency rather than sharding model weights across GPUs.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Inference Stream Parallelism. Refine with filters or upvote what's useful.
Awesome Inference Stream Parallelism GitHub Repositories
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- wang-xinyu/tensorrtx
wang-xinyu/tensorrtx
7,802Vezi pe GitHubtensorrtx is a computer vision inference engine and model implementation library designed for graphics processor acceleration. It provides a framework for optimizing deep learning models through a GPU inference optimizer, a deep learning model converter for transforming weights from frameworks like TensorFlow and PyTorch, and a custom plugin library to implement operations not natively supported by the TensorRT API. The project distinguishes itself through a comprehensive collection of pre-defined network implementations, ranging from various YOLO versions and DETR transformers for object det
Distributes inference tasks across multiple hardware contexts and non-blocking streams to increase total device throughput.
C++arcfacecrnndetr
