Awesome GitHub RepositoriesLLM Development Frameworks

GPT4All is a cross-platform runtime environment designed to execute large language models directly on local consumer hardware. By leveraging an optimized C++ inference backend, it enables private, offline AI interactions without requiring an internet connection or external cloud services. The project provides a comprehensive ecosystem for managing the entire model lifecycle, including discovery, downloading, and configuration of local weights. What distinguishes the platform is its integrated retrieval-augmented generation engine, which allows users to index local documents into semantic vect

Unsloth is a high-performance training and inference platform designed to optimize the lifecycle of large language and multimodal models. It provides a comprehensive engine for fine-tuning, executing, and managing models locally, with a focus on reducing memory consumption and increasing compute speed on consumer-grade hardware. The platform distinguishes itself through hand-optimized kernels and automated computational graph techniques that maximize hardware throughput. It supports advanced training methodologies, including reinforcement learning for reasoning and efficient adapter-based fin

This project is a data processing engine and AI application platform designed for building production-grade machine learning workflows. It provides a unified programming model that handles both historical batch data and live stream ingestion, enabling the development of real-time ETL pipelines and scalable data transformation workflows. The framework distinguishes itself through differential dataflow execution, which propagates only changes through a pipeline rather than recomputing entire datasets. It supports distributed state management across worker nodes and utilizes incremental stream p

Langfuse is an open-source observability and evaluation platform designed for language model applications. It provides a centralized system for tracking execution traces, monitoring performance metrics, and managing prompt templates. By capturing hierarchical units of work and telemetry data, the platform enables developers to debug complex application lifecycles and analyze token usage, latency, and model interactions in production environments. The platform distinguishes itself through an integrated evaluation framework that allows for systematic benchmarking and automated scoring of model

Vanna is a Python framework designed to build conversational interfaces that translate natural language into executable database queries. It functions as an enterprise-grade toolkit that connects language models to relational databases, allowing users to retrieve information through conversational prompts rather than manual code. The system maintains context across interactions by utilizing vector databases to store historical query patterns and schema metadata. The framework distinguishes itself through a focus on security and schema-aware generation. It incorporates granular access control,

Opik is an observability and evaluation platform designed for generative AI applications and agentic workflows. It provides a centralized environment for tracing execution flows, managing prompt templates, and monitoring production performance, allowing teams to gain visibility into complex model interactions and tool usage without requiring manual application code changes. The platform distinguishes itself through its integrated approach to the AI development lifecycle, combining distributed trace instrumentation with automated evaluation frameworks. It supports model-as-a-judge scoring, syn

Botpress is a conversational AI builder and LLM agent platform used to design chatbot workflows and orchestrate agents powered by large language models. It provides a framework for managing the entire lifecycle of these agents, from initial creation through to deployment across various production environments. The platform includes a custom integration SDK for developing and publishing third-party connectors that extend agent capabilities. These tools allow for the creation of custom plugins that connect AI agents to external APIs and third-party services. The system supports both visual des

WrenAI is a platform designed to enable natural language interaction with relational and analytical databases. By combining a text-to-SQL engine with semantic data modeling, it allows users to explore structured data through plain language questions, removing the requirement for manual code generation. The system functions by grounding natural language requests in a predefined business logic layer rather than raw database schemas. This semantic approach, supported by context-aware prompt engineering, ensures that generated queries remain consistent and accurate across an organization. The pla

Ludwig is a multimodal machine learning platform and low-code framework designed for building, training, and deploying neural networks. It enables the construction of models that process text, images, audio, and tabular data through a unified interface using declarative configuration files rather than custom code. The system features a specialized low-code framework for large language models, supporting supervised fine-tuning, preference alignment, and a constrained decoding tool to force structured data output via logit extraction. It also includes an automated model architecture search to i

Promptflow is a development framework and orchestrator for building applications powered by large language models. It functions as a suite of tools for designing, orchestrating, and deploying AI workflows by linking prompts, custom Python code, and language models into executable sequences. The project is distinguished by a visual AI workflow designer that allows for the creation of directed acyclic graphs of logic nodes. It provides a dedicated prompt engineering environment for versioning and comparing templates, alongside stateful execution tracing to record function calls and variable val

Promptfoo is an evaluation framework designed for testing, benchmarking, and red-teaming language models and agentic workflows. It provides a unified environment to run prompts against multiple providers, allowing developers to systematically validate model outputs against objective assertions, semantic similarity metrics, and custom grading rubrics. The platform distinguishes itself through a provider-agnostic execution layer and a stateful orchestrator capable of simulating multi-turn conversations and complex tool-use trajectories. It includes a dedicated adversarial mutation pipeline that

Model Context Protocol is a standardized framework for connecting large language models to external data sources and executable tools. It enables the creation of a universal interface where servers expose tools, resources, and prompts that can be discovered and utilized by various AI clients. The protocol utilizes a JSON-RPC message system that is transport-agnostic, supporting both standard input/output for local processes and HTTP with server-sent events for remote connections. It emphasizes security and control by delegating model sampling to the client to keep API keys secure from servers

Manifest is a language model provider unification system that standardizes access to multiple AI backends through a single interface. It functions as a centralized management layer for integrating various cloud-based and local model providers to simplify how applications request completions. The system provides intelligent model routing and high availability infrastructure by directing queries based on complexity and automatically triggering model fallbacks when a primary provider fails. It distinguishes itself through multi-tenant AI management, organizing agents into isolated groups with de

Swarms est un framework d'orchestration multi-agents et une boîte à outils d'agents autonomes conçue pour coordonner des agents de grands modèles de langage. Il sert de moteur de flux de travail pour gérer les relations entre agents, fournissant l'infrastructure pour construire des agents autonomes avec une mémoire intégrée, des capacités d'appel d'outils et des boucles de raisonnement. Le framework se distingue par ses systèmes de consensus multi-agents, qui utilisent le vote, les débats contradictoires et des agents juges pour synthétiser des réponses de haute qualité. Il prend en charge une variété de modèles de collaboration, y compris les hiérarchies directeur-travailleur, la synthèse d'experts et la génération d'architecture d'essaim automatisée basée sur des descriptions en langage naturel. Le système couvre un large éventail de capacités opérationnelles, y compris l'orchestration de flux de travail basée sur des graphes et séquentielle via un langage spécifique au domaine, une interface unifiée pour divers fournisseurs de modèles et l'intégration avec le protocole de contexte de modèle pour la découverte dynamique d'outils. Il inclut également un support pour la génération augmentée par récupération, la mémoire persistante d'état et la capacité d'exposer la fonctionnalité d'agent en tant que services web. Le projet fournit une interface en ligne de commande pour la gestion des agents et prend en charge la configuration via YAML et des fichiers de compétences markdown modulaires.

Steel is a cloud browser automation platform that provides a REST API for launching and controlling remote Chrome browser sessions. It enables programmatic browsing and web scraping using standard automation tools like Puppeteer, Playwright, and Selenium, connecting to cloud-hosted browser instances via WebSocket and the Chrome DevTools Protocol. The platform supports both headless and headful browser sessions, with language-specific SDKs for TypeScript and Python. The service distinguishes itself through comprehensive anti-detection capabilities, including residential proxy rotation, CAPTCHA

Helicone is an AI gateway and observability platform designed to intercept, manage, and monitor interactions with large language models. By acting as a reverse-proxy, it provides a centralized layer for routing requests across multiple AI providers, allowing developers to maintain consistent application logic while gaining deep visibility into model performance, usage, and costs. The platform distinguishes itself through a robust suite of traffic management and prompt engineering tools. It enables policy-driven control, including automatic failover between providers, rate limiting, and edge-b

Plano is an AI agent orchestrator and LLM gateway proxy that unifies access to multiple AI providers through a single interoperable interface. It functions as a model routing engine that decouples applications from specific vendors using semantic aliases, allowing traffic to be shifted between providers without modifying application code. The system distinguishes itself with intent-based agent routing, which directs prompts to specialized agents based on semantic analysis. It features an interceptor-based filter chain system that acts as guardrail middleware to enforce safety policies, rewrit

Agenta is a Prompt Ops lifecycle manager and prompt management platform that decouples prompt engineering from application code. It serves as a centralized system for developing, versioning, and deploying prompt templates and model configurations across different environments. The platform functions as an AI agent orchestrator with a visual interface for building agent workflows and connecting models to external tools. It further acts as an evaluation framework and observability tool, utilizing OpenTelemetry to capture execution traces, monitor latency, and track token costs. The system cove

Open source platform for AI Engineering: OpenTelemetry-native LLM Observability, GPU Monitoring, Guardrails, Evaluations, Prompt Management, Vault, Playground. 🚀💻 Integrates with 50+ LLM Providers, VectorDBs, Agent Frameworks and GPUs.