How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
Nano-vllm is a high-performance inference engine designed for executing large language models locally. It functions as a specialized runtime that prioritizes accelerated token generation and efficient hardware utilization for text generation tasks. The project distinguishes itself through a comprehensive suite of optimization techniques, including a graph compilation engine that transforms neural network operations into pre-compiled execution plans. It also incorporates a tensor parallelism framework to distribute model weights across multiple hardware accelerators, effectively reducing memor
The main features of jingyaogong/minimind-o are: Small Multimodal Models, Small Vision Models.
Projects with overlapping indexed features include: ai-study-han/zero-qwen-vl. coobiw/mpp-llava. emericen/tiny-qwen. geeeekexplorer/nano-vllm — Nano-vllm is a high-performance inference engine designed for executing large language models locally. It functions as… huggingface/nanovlm — nanoVLM is a training framework and toolkit for small vision-language models. It provides a PyTorch-based environment… jingyaogong/minimind-v.