1 Repo
Awesome GitHub RepositoriesDecoder-Only Inference Modes
Inference configurations that load only the decoder portion of a speech model to reduce memory and computation.
Distinct from Prefill-Decode Disaggregation: Distinct from Prefill-Decode Disaggregation: focuses on using only the decoder part, not separating prefill and decode phases.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Decoder-Only Inference Modes. Refine with filters or upvote what's useful.
Awesome Decoder-Only Inference Modes GitHub Repositories
Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
- neuphonic/neutts
neuphonic/neutts
6,007Auf GitHub ansehenNeutts is a neural text-to-speech engine designed for real-time streaming output on edge devices such as phones and laptops. It supports voice cloning from short audio references, enabling zero-shot reproduction of a target speaker's voice, and can be fine-tuned or retrained from scratch for custom voices and styles. The system distinguishes itself through a decoder-only architecture that halves memory and accelerates generation on constrained hardware, combined with quantized model inference for reduced memory footprint. Its streaming decoder loop interleaves synthesis with playback, deliver
Loads only the decoder portion of the speech model during inference to minimize memory and computation.
Python
