SJTU-IPADS/PowerInfer

Features

• Inference Frameworks - Fast serving optimized for consumer-grade hardware.
• Local LLM Execution - High-speed inference engine for deploying models locally.
• Model Pruning and Sparsity - Enables fast serving on consumer-grade hardware.
• Model Serving & Deployment - Leverages activation locality for CPU/GPU LLM inference.