ModelTC/LightLLM

Features

• Inference Frameworks - Python-based framework optimized for high-performance serving.
• Model Serving & Deployment - Provides a lightweight, high-speed LLM inference framework.
• Inference Frameworks - Lightweight inference framework with efficient KV cache management.