Projects sharing features with Tiny Qwen

Nano-vllm is a high-performance inference engine designed for executing large language models locally. It functions as a specialized runtime that prioritizes accelerated token generation and efficient hardware utilization for text generation tasks. The project distinguishes itself through a comprehensive suite of optimization techniques, including a graph compilation engine that transforms neural network operations into pre-compiled execution plans. It also incorporates a tensor parallelism framework to distribute model weights across multiple hardware accelerators, effectively reducing memor

nanoVLM is a training framework and toolkit for small vision-language models. It provides a PyTorch-based environment for training and fine-tuning models to associate image inputs with textual descriptions and generate natural language answers. The project includes a cloud model versioning tool for saving and loading model weights to centralized repositories to synchronize assets across environments. It also features a dedicated evaluation suite for measuring the accuracy and reliability of vision-language models against standard task datasets. The framework covers GPU resource planning thro