1 repo
Awesome GitHub RepositoriesInference Benchmarking Tools
Utilities for measuring processing speed, latency, and performance metrics of machine learning models across various hardware configurations.
Distinguishing note: Focuses specifically on performance measurement and latency analysis for ML inference, distinct from general model training or deployment frameworks.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Inference Benchmarking Tools. Refine with filters or upvote what's useful.
Awesome Inference Benchmarking Tools GitHub Repositories
Find the best repos with AI.We'll search the best matching repositories with AI.
- ggml-org/whisper.cpp
ggml-org/whisper.cpp
46,843View on GitHubWhisper.cpp is a high-performance, local-first speech recognition engine designed to run large-scale machine learning models on consumer hardware. It functions as a portable library that converts audio into text, supporting both static file transcription and real-time stream processing. By utilizing a lightweight inference engine and weight quantization, the project minimizes memory and compute overhead, allowing for efficient execution without reliance on external cloud APIs or internet connectivity. The project distinguishes itself through a hardware-agnostic compute abstraction that offloa
Measures processing speed and latency across hardware configurations to determine performance for speech recognition tasks.
C++inferenceopenaispeech-recognition
