LiteLLM is a unified gateway and proxy server designed to centralize access to over one hundred language model providers. It provides a standardized API interface that abstracts vendor-specific schemas, allowing developers to interact with diverse models through a single, consistent format. By acting as a central traffic management layer, it enables organizations to route, secure, and govern model interactions across multiple deployments. The platform distinguishes itself through its policy-driven architecture, which uses configuration-based routing to manage traffic distribution, load balanc
StarCoder2 is a family of code generation models (3B, 7B, and 15B), trained on 600+ programming languages from The Stack v2 and some natural language text such as Wikipedia, Arxiv, and GitHub issues. The models use Grouped Query Attention, a context window of 16,384 tokens, with sliding window…
This project is a quantized fine-tuning framework for large language models. It implements a low-rank adaptation library and a four-bit quantizer to reduce the GPU memory requirements needed to train large models. The framework utilizes four-bit quantization and low-rank adapters to enable model training on consumer-grade hardware. It further reduces the memory footprint through double quantization and a paged optimizer that offloads states to system RAM. The system supports distributed training across multiple GPUs to handle larger parameter scales and includes utilities for custom dataset
Petals is a decentralized framework and inference engine for running large language models across a peer-to-peer network. It enables the execution of models that exceed the memory of any single machine by splitting computations and model layers across a collaborative swarm of GPUs. The system functions as a collaborative compute network where participants share local GPU resources and host model weights. It supports distributed prompt-tuning to adapt massive models to specific tasks and allows for the establishment of private compute swarms to process sensitive data within restricted, trusted
PyTorch extensions for high performance and large scale training.
facebookresearch/fairscale 的主要功能包括：Large Language Models, Large Language Models (LLMs)。
facebookresearch/fairscale 的开源替代品包括： bigscience-workshop/petals — Petals is a decentralized framework and inference engine for running large language models across a peer-to-peer… chroma-core/chroma — Chroma is a specialized vector database designed to index and retrieve high-dimensional data representations for… berriai/litellm — LiteLLM is a unified gateway and proxy server designed to centralize access to over one hundred language model… bigcode-project/starcoder2 — StarCoder2 is a family of code generation models (3B, 7B, and 15B), trained on 600+ programming languages from The… artidoro/qlora — This project is a quantized fine-tuning framework for large language models. It implements a low-rank adaptation… databrickslabs/dolly — Dolly is an instruction-tuned large language model designed to follow complex natural language directions. It operates…