Open-source alternatives to Dspy

AdalFlow is an autonomous AI agent framework and LLM application library designed for building modular workflows. It serves as a model-agnostic interface and RAG pipeline orchestrator, allowing users to develop ReAct agents that utilize iterative reasoning and external tool execution to solve complex tasks. The project distinguishes itself through a prompt optimization system that uses textual gradient descent to automatically refine prompt templates and few-shot examples. It treats model feedback as a differentiable signal, enabling a form of LLM backpropagation to iteratively improve output

Kiln is an LLM development workbench and evaluation framework designed for designing, testing, and optimizing prompts and AI agents. It functions as a multi-agent orchestrator and a RAG optimization tool, providing a visual interface for the iterative development of AI systems. The project distinguishes itself through a comprehensive fine-tuning pipeline that supports zero-code model training and reasoning distillation. It enables the creation of hierarchical multi-agent systems where specialized actors coordinate via tool calling, and it implements a Model Context Protocol server to expose t

LlamaIndex is a comprehensive development framework designed to connect private or external data sources to large language models. It functions as a data-centric toolkit that enables the construction of retrieval-augmented generation systems, allowing developers to build applications that provide context-aware answers based on specific organizational information. The project distinguishes itself through a robust agentic orchestration engine that supports the creation of autonomous agents capable of multi-step reasoning, memory management, and complex tool execution. Beyond simple retrieval, i

Mastra is an orchestration framework designed for building, deploying, and managing autonomous AI agents and multi-agent systems. It provides a comprehensive suite of primitives for creating resilient AI applications, including durable workflow orchestration, event-driven agent loops, and semantic memory management. By integrating these core components, the platform enables developers to build complex, multi-step processes that can reason about goals and execute tasks without manual intervention. The framework distinguishes itself through its focus on observability and secure, isolated execut

PydanticAI is a Python framework designed for building production-grade autonomous agents. It provides a unified interface for interacting with diverse language models, enabling developers to construct agents that perform complex tasks through structured data validation, tool execution, and multi-turn conversation management. The library centers on type-safe schema enforcement, ensuring that model inputs and outputs remain consistent and reliable throughout the agent's lifecycle. The framework distinguishes itself through a robust architecture that emphasizes modularity and testability. It ut

Dify is an open-source platform for building, orchestrating, and deploying generative AI applications and autonomous agents. It provides a visual development environment that allows users to design complex, multi-step logic chains and conversational flows, which can then be published as APIs, web interfaces, or embedded widgets. The platform acts as a centralized infrastructure layer, managing model connections, prompt templates, and knowledge retrieval to support scalable AI-powered services. What distinguishes the platform is its focus on stateful application design and workflow orchestrati

This framework provides a development environment for building collaborative systems where autonomous agents interact to solve complex tasks through conversational workflows. It functions as a conversational workflow engine and event-driven runtime, coordinating multi-step processes by translating high-level goals into structured dialogue sequences between specialized agents. The system distinguishes itself through its message-passing orchestration, which manages state transitions and task delegation between independent participants. It supports dynamic conversation state management to provid

LangChain is a framework for building applications that chain large language models with external data sources and third-party tools. It serves as an orchestrator for autonomous agents that use language models to plan and execute multi-step tasks, while providing a toolkit for linking interoperable AI components into sequences to prototype complex model behaviors. The project provides a model agnostic integration layer, allowing users to switch between different language model providers using a standardized interface. It also includes tools for observability and evaluation to track the perfor

PromptWizard is an automated prompt engineering framework designed to evolve natural language instructions for generative tasks. It functions as an in-context learning optimizer and synthetic data generator, using mutation rounds and performance metrics to iteratively refine large language model instructions. The system employs a self-reflective optimization loop that uses model-generated critiques to rewrite prompts. It distinguishes itself through the use of reasoning chain integration and persona-based prompting to steer the tone and professional quality of model responses. The framework

Automatic Prompt Engineer is a framework designed to automate the generation, refinement, and performance measurement of language model instructions. It functions as a systematic tool for optimizing prompt phrasing by iteratively testing candidate instructions against specific input and output datasets to maximize task accuracy. The system distinguishes itself through an evaluation-driven approach that uses automated feedback loops to score prompt variations. By employing template-based input structuring, it ensures consistent testing environments where candidate instructions are measured aga

Promptfoo is an evaluation framework designed for testing, benchmarking, and red-teaming language models and agentic workflows. It provides a unified environment to run prompts against multiple providers, allowing developers to systematically validate model outputs against objective assertions, semantic similarity metrics, and custom grading rubrics. The platform distinguishes itself through a provider-agnostic execution layer and a stateful orchestrator capable of simulating multi-turn conversations and complex tool-use trajectories. It includes a dedicated adversarial mutation pipeline that

Ragas is an evaluation framework designed to measure the performance of retrieval-augmented generation pipelines and autonomous agent workflows. It provides a comprehensive suite of tools for benchmarking system outputs, utilizing language models as automated judges to score performance against defined rubrics and reference data. By standardizing inputs, retrieved contexts, and generated responses into a unified schema, the project enables consistent analysis across complex AI applications. The framework distinguishes itself through its ability to generate synthetic test datasets from existin

txtai is an artificial intelligence platform designed for building semantic search applications, managing vector storage, and orchestrating language model workflows. It functions as a comprehensive engine for processing unstructured data, enabling the development of autonomous agents and complex content automation pipelines. The platform distinguishes itself through a hybrid indexing architecture that combines dense vector embeddings with relational graph structures, allowing for multi-dimensional retrieval across both semantic meaning and entity relationships. It supports multimodal analysis

TensorZero is an inference gateway and experimentation framework designed to manage the lifecycle of large language models in production environments. It functions as a central proxy that routes requests across multiple artificial intelligence providers while providing the infrastructure necessary to monitor performance, track costs, and ensure service reliability. The platform distinguishes itself by integrating a comprehensive evaluation engine and an observability pipeline directly into the request flow. It enables developers to conduct controlled experiments and A/B tests to compare diffe

Promptflow is a development framework and orchestrator for building applications powered by large language models. It functions as a suite of tools for designing, orchestrating, and deploying AI workflows by linking prompts, custom Python code, and language models into executable sequences. The project is distinguished by a visual AI workflow designer that allows for the creation of directed acyclic graphs of logic nodes. It provides a dedicated prompt engineering environment for versioning and comparing templates, alongside stateful execution tracing to record function calls and variable val

LangGraph is a framework for building stateful, multi-step agentic workflows by modeling application logic as a directed graph. It provides a runtime environment where complex tasks are orchestrated through interconnected nodes and edges, allowing developers to manage state transitions, persistent memory, and control flow across long-running automated processes. The platform distinguishes itself through its native support for human-in-the-loop automation, enabling developers to define breakpoints that pause execution for manual review, modification, or approval. It also features checkpoint-ba

The foundation layer for agentic intelligence.

Mem0 is an agent-agnostic memory layer designed to provide intelligent agents with long-term persistence and cross-session state management. By acting as a centralized service, it allows diverse AI agents to recall user preferences, past interactions, and historical context, ensuring continuity across multiple workflows and independent agent systems. The platform distinguishes itself through a multi-signal retrieval engine that combines semantic vectors, keyword matching, and entity-linked metadata to surface the most relevant information. It employs an adaptive memory engine that automatical

This project is an automated prompt engineering and optimization tool designed to iteratively create, test, and refine prompts using a language model to improve output quality. It functions as a framework for generating candidate prompts and ranking their performance through correctness matching and ELO-based ratings. The system includes capabilities for model distillation, generating high-quality example pairs from frontier models to create training data for smaller models. It also provides tools to condense prompts for smaller models and transform instruction-tuned prompts into completion-b

This project is a framework for the iterative optimization and validation of LLM agent skills. It functions as an agent capability orchestrator and prompt optimizer, utilizing an evaluation framework to measure performance through weighted rubrics and automated rewriting. The system distinguishes itself through a closed-loop optimization cycle that employs independent reviewer agents to prevent anchoring effects and a ratchet-based version control mechanism that automatically reverts changes if they fail to improve baseline scores. It also features exploratory structural rewriting to overcome

This project provides a comprehensive framework for building, training, and managing autonomous agents. It enables the construction of systems that utilize language models to plan, manage memory, and execute multi-step tasks through iterative reasoning loops and tool-based actions. The framework distinguishes itself by offering specialized capabilities for interacting with graphical user interfaces and legacy software, allowing agents to perceive visual elements and perform actions like a human user. It supports complex, cross-application workflows through graph-based orchestration and provid

AReaL is a system for agent orchestration, distributed model training, and parameter-efficient tuning. It provides a framework for developing multi-turn reasoning agents and training large models using reinforcement learning from human feedback. The project implements a toolkit for improving the visual reasoning and geometry problem solving capabilities of vision-language models. It utilizes a memory-efficient tuning system to optimize mathematical and reasoning models across different inference backends. The infrastructure supports large-scale training through tensor, pipeline, and expert p

Evidently is an AI observability platform and evaluation framework designed to quantify the performance of machine learning models and large language models. It functions as a monitoring tool for detecting data drift and quality degradation in tabular datasets, while providing a specialized analyzer for the faithfulness and correctness of retrieval augmented generation systems. The project distinguishes itself through an evaluation framework that utilizes judge models and custom rubrics to score language model outputs. It includes tools for iterative prompt optimization and the generation of

This project is a comprehensive educational resource and technical guide focused on the development, optimization, and application of large language models. It provides a structured curriculum for mastering prompt engineering, ranging from foundational principles of instruction design to advanced techniques for improving model reasoning, accuracy, and reliability. The guide distinguishes itself by offering deep technical insights into agentic workflows and autonomous system design. It covers the implementation of multi-step reasoning chains, tool integration through function calling, and stat

Tiny Universe is an educational monorepo that delivers multiple independent implementations of core AI subsystems as self-contained Jupyter notebooks. It provides from-scratch constructions of foundational architectures including a complete Transformer model built from the original paper specification, a denoising diffusion probabilistic model for image generation, and a ReAct-style autonomous agent framework that equips an LLM with tools for planning and multi-step task execution. The project distinguishes itself by covering the full lifecycle of modern AI systems through hands-on implementa

Promptbase is a prompt engineering framework designed for designing, testing, and optimizing prompts for large language models. It provides a system for measuring model accuracy and performance through an evaluation toolkit that compares outputs against ground-truth datasets. The project also includes an orchestration pipeline for automating multi-component machine learning tasks across cloud-based endpoints and a utility for preparing retrieval-augmented generation datasets. The framework distinguishes itself through advanced response quality optimization, utilizing chain-of-thought generato

TypeChat is a schema enforcement library and framework for building natural language interfaces. It ensures that responses from large language models strictly adhere to predefined TypeScript type definitions, translating unstructured human language into predictable, structured data. The project functions as both a prompt generator and an output validator. It automatically creates model instructions by extracting requirements from type schemas to replace manual prompt engineering and verifies that model outputs match the required format. The system handles structured output generation and res