turboderp/exllama

Features

• Inference and Serving - Memory-efficient implementation for running quantized Llama models.
• Inference Engines - Memory-efficient inference implementation optimized for GPU execution.