FMInference/FlexGenArchived

Features

• Hardware Optimized Inference - High-throughput generative inference optimized for single GPU environments.
• Inference Frameworks - High-throughput generative inference on single-GPU systems.
• Model Serving Engines - Throughput-oriented inference engine for running models on single GPUs.