1 repository
Awesome GitHub RepositoriesGenerative Text Decoding
The process of producing human-readable text sequences from prompts using prefill and decoding loops.
Distinguishing note: Existing candidates are for specific dialogue generators or TTS; this is the general LLM text generation loop.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Generative Text Decoding. Refine with filters or upvote what's useful.
Awesome Generative Text Decoding GitHub Repositories
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- skyzh/tiny-llm
skyzh/tiny-llm
4,304Vezi pe GitHubtiny-llm is a large language model inference engine and transformer model implementation. It serves as a quantized model runtime and paged key-value cache manager, providing a specialized inference stack optimized for Apple Silicon. The system distinguishes itself through high-throughput execution techniques, including continuous batching and paged attention. It utilizes a paged memory system to eliminate fragmentation during token generation and employs on-the-fly dequantization of compressed weights to reduce the memory footprint during matrix multiplication. The project covers a broad ran
Produces sequences of tokens from a prompt using a prefill stage and a decoding loop.
Pythoncourselarge-language-modelllm
