Projects sharing features with Dust

LangChain is an orchestration framework designed for building, managing, and deploying applications powered by large language models. It provides a unified integration layer that normalizes disparate model provider APIs into a consistent set of primitives, enabling developers to build complex, multi-step AI workflows that manage state, memory, and tool execution. The project distinguishes itself through a durable execution runtime that maintains persistent state across long-running processes by checkpointing progress to external storage. It models agent workflows as directed graphs, allowing

The foundation layer for agentic intelligence.

This repository is a deep learning for natural language processing course and curriculum. It provides educational material and guides focused on neural network architectures used for processing natural language, speech signals, and text classification. The content includes instructional tutorials on sequence modeling and neural language modeling, covering the implementation of n-gram and recurrent neural networks. It also provides a framework for studying word embeddings to map linguistic meanings into numerical representations. The curriculum covers a broad range of capabilities, including

Fructose is a python package to create a dependable, strongly-typed interface around an LLM call.

Agents Capable of Self-Editing Their Prompts / Python Code

This project is a framework for managing generative AI services through a unified provider interface and adapter layer. It provides a standardized API for calling multiple cloud-based and locally hosted models, translating provider-specific parameters and responses into a uniform format. The system includes an agent orchestrator designed for long-running tasks, featuring state persistence for resuming runs and execution tracing to monitor decision-making processes. It integrates the Model Context Protocol to connect models to external servers and filesystems and employs a policy-based executi

Eino is an AI agent development kit and LLM application framework designed for building autonomous agents and orchestrating complex language model workflows. It serves as a multi-agent orchestration engine and workflow orchestrator, providing a graph-based execution model to route data between models, tools, and retrievers. The framework distinguishes itself through a robust set of multi-agent coordination patterns, including supervisor-led management, sequential flows, and autonomous reasoning loops like ReAct. It features advanced agent execution controls such as active turn preemption, che

Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, trace, persist, and execute on your own infrastructure.

Transformers is a comprehensive library for machine learning that provides a unified interface for training, fine-tuning, and deploying transformer-based models. It supports a wide range of tasks, including text classification, language modeling, question answering, and sequence-to-sequence translation, while offering specialized architectures for both text and vision processing. The framework includes tools for managing the entire model lifecycle, from data preprocessing and tokenization to distributed training and inference. The library features extensive support for model optimization and

A dead-simple API to build LLM-powered apps

Lightweight LLM Interaction Framework

Casibase is an open-source platform that orchestrates multi-turn conversations with large language models and manages retrieval-augmented knowledge bases from a single interface. It provides a unified system for connecting to over 30 AI model providers, ingesting documents into vector embeddings for semantic search, and running autonomous agent loops that can drive a browser, search the web, execute commands, and integrate with external tools. The platform distinguishes itself by combining AI conversation management with infrastructure and application orchestration capabilities. It includes a

The TypeScript framework for AI development

Chaining AI & API agents to streamline software development and achieve goals collaboratively.

spaCy is a Python natural language processing framework designed for industrial-scale text processing. It converts raw text into structured data for machine learning pipelines through a combination of statistical language model trainers, transformer-based text processors, and syntactic dependency parsers. The project enables the integration of pretrained transformer architectures to perform complex linguistic analysis and multi-task learning. It also provides a specialized system for neural named entity recognition to identify and categorize key entities within text. The framework covers a b

The AI Application Framework for Javascript

Flair is a transformer-based natural language processing framework used to build and train models for text classification and sequence tagging. It provides a specialized library for generating contextual text embeddings and performing linguistic analysis. The framework includes dedicated tools for named entity recognition, including the identification of specialized biomedical entities across multiple languages. It further supports entity linking to map identified text mentions to unique entries within general or biomedical knowledge bases. The project covers a broad range of language analys

End-to-end speech recognition large model: 31 languages, dialects, accents, lyrics, hotwords, timestamps, speaker diarization. Trained on tens of millions of hours.

Griptape is a Python framework for building generative AI applications, autonomous agents, and complex AI workflows. It functions as both an AI agent orchestrator and a workflow engine, capable of managing sequential pipelines and directed acyclic graphs to ensure predictable execution of AI tasks. The framework distinguishes itself through a focus on security and governance, utilizing a Docker-based environment to execute model-generated code and shell commands in isolation. It employs a driver-based abstraction layer that allows developers to swap language model providers and vector stores

h2oGPT is a self-hosted platform designed for running large language models and executing retrieval-augmented generation workflows locally. It provides a comprehensive web interface that allows users to index private document collections into searchable databases, enabling context-aware question answering and summarization without exposing sensitive data to external services. The platform distinguishes itself by offering a modular architecture that supports both local model execution and connections to external inference servers. It facilitates the development of autonomous agents capable of

LLaMA-Factory is a comprehensive suite for dataset preparation, model fine-tuning, memory optimization, and standardized API deployment. It provides a unified platform for the supervised and reward-based fine-tuning of large language models and vision-language models. The framework includes a specialized toolkit for training vision-language models and a model serving interface that deploys trained models through high-performance APIs. It utilizes precision tuning and quantization techniques to reduce the hardware requirements and memory footprint of large models. The system covers data pipel

Botpress is a conversational AI builder and LLM agent platform used to design chatbot workflows and orchestrate agents powered by large language models. It provides a framework for managing the entire lifecycle of these agents, from initial creation through to deployment across various production environments. The platform includes a custom integration SDK for developing and publishing third-party connectors that extend agent capabilities. These tools allow for the creation of custom plugins that connect AI agents to external APIs and third-party services. The system supports both visual des

LangChain is a framework for building applications that chain large language models with external data sources and third-party tools. It serves as an orchestrator for autonomous agents that use language models to plan and execute multi-step tasks, while providing a toolkit for linking interoperable AI components into sequences to prototype complex model behaviors. The project provides a model agnostic integration layer, allowing users to switch between different language model providers using a standardized interface. It also includes tools for observability and evaluation to track the perfor

Rivet is a visual LLM workflow designer and AI agent orchestration engine. It serves as a development environment for building retrieval augmented generation pipelines and a TypeScript library for embedding visual AI graphs and prompt logic into JavaScript applications. The system differentiates itself through a node-based editor that maps data flow between language models, vector databases, and external APIs. It provides specialized tools for prompt engineering, including interfaces for iterative prompt refinement and A/B testing to improve model response quality. The platform covers a broa

Seamlessly integrate LLMs as Python functions

LMQL is a programming language and probabilistic interface that blends algorithmic logic with stochastic text generation. It functions as a constraint-guided prompting framework and structured output generator, allowing users to force model responses to adhere to strict formatting and data types. The system distinguishes itself as an inference optimizer that increases token throughput and reduces latency. This is achieved through specialized execution strategies, including tree-based prompt caching and asynchronous batch processing. The project covers a broad range of generation control capa