How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
Home of CodeT5: Open Code LLMs for Code Understanding and Generation
Petals is a decentralized framework and inference engine for running large language models across a peer-to-peer network. It enables the execution of models that exceed the memory of any single machine by splitting computations and model layers across a collaborative swarm of GPUs. The system functions as a collaborative compute network where participants share local GPU resources and host model weights. It supports distributed prompt-tuning to adapt massive models to specific tasks and allows for the establishment of private compute swarms to process sensitive data within restricted, trusted
This project is a quantized fine-tuning framework for large language models. It implements a low-rank adaptation library and a four-bit quantizer to reduce the GPU memory requirements needed to train large models. The framework utilizes four-bit quantization and low-rank adapters to enable model training on consumer-grade hardware. It further reduces the memory footprint through double quantization and a paged optimizer that offloads states to system RAM. The system supports distributed training across multiple GPUs to handle larger parameter scales and includes utilities for custom dataset
LiteLLM is a unified gateway and proxy server designed to centralize access to over one hundred language model providers. It provides a standardized API interface that abstracts vendor-specific schemas, allowing developers to interact with diverse models through a single, consistent format. By acting as a central traffic management layer, it enables organizations to route, secure, and govern model interactions across multiple deployments. The platform distinguishes itself through its policy-driven architecture, which uses configuration-based routing to manage traffic distribution, load balanc
StarCoder2 is a family of code generation models (3B, 7B, and 15B), trained on 600+ programming languages from The Stack v2 and some natural language text such as Wikipedia, Arxiv, and GitHub issues. The models use Grouped Query Attention, a context window of 16,384 tokens, with sliding window…
The main features of bigcode-project/starcoder2 are: Large Language Models, Pre-training Research, Large Language Models (LLMs).
Projects with overlapping indexed features include: salesforce/codet5 — Home of CodeT5: Open Code LLMs for Code Understanding and Generation. databrickslabs/dolly — Dolly is an instruction-tuned large language model designed to follow complex natural language directions. It operates… berriai/litellm — LiteLLM is a unified gateway and proxy server designed to centralize access to over one hundred language model… bigscience-workshop/petals — Petals is a decentralized framework and inference engine for running large language models across a peer-to-peer… artidoro/qlora — This project is a quantized fine-tuning framework for large language models. It implements a low-rank adaptation… chroma-core/chroma — Chroma is a specialized vector database designed to index and retrieve high-dimensional data representations for…