Open-source alternatives to Enchanted

Streamdown is a streaming markdown renderer for React that transforms incoming markdown text into sanitized HTML in real time, handling incomplete blocks as they arrive. It parses GitHub-Flavored Markdown syntax including tables, task lists, and footnotes, and renders LaTeX math expressions using KaTeX, Mermaid diagrams as interactive SVGs, and code blocks with Shiki-based syntax highlighting supporting over 200 languages with dual light and dark themes. The renderer includes an XSS-safe HTML sanitizer that strips dangerous tags and validates URLs to prevent injection attacks. What distinguis

MovieSwiftUI is a movie discovery application built with SwiftUI that integrates with the MovieDB API to retrieve and display movie information, ratings, and metadata. It functions as a cross-platform Apple application, providing a consistent user experience across iOS and macOS from a single codebase. The project implements a reactive data flow using Combine to synchronize global application state with the user interface. It employs a unidirectional data flow and a centralized store to maintain a single source of truth across different screens and components. The application utilizes declar

Ollama is a cross-platform runtime for managing, serving, and executing large language models on local hardware. It functions as a model manager and orchestrator that allows for the downloading, updating, and organization of model weights and configurations to ensure private and offline inference. The system provides a local inference API and a RESTful interface for programmatic model lifecycle management and text generation. It utilizes a compiled C++ backend to handle tensor operations and memory management. To support various hardware configurations, the runtime employs dynamic GPU offloa

omlx is a local inference server designed to run large language models, vision models, and embedding models on Apple Silicon. It provides a private alternative to industry-standard AI endpoints by hosting a local API gateway that mirrors OpenAI and Anthropic specifications. The system distinguishes itself through specialized hardware optimizations, including continuous batching for high throughput and a tiered caching system that offloads memory blocks to SSD. It also functions as a Model Context Protocol host, enabling the integration of local models with external tools, agents, and structur

Chat with your favourite LLaMA models in a native macOS app

Ktransformers is a comprehensive framework designed for the operation, fine-tuning, and serving of large language models. It functions as a heterogeneous inference engine and quantized execution runtime, enabling the deployment of massive models by distributing computational workloads across both CPU and GPU resources. This architecture allows users to bypass local memory constraints, making it possible to run and train models that exceed the capacity of a single device. The project distinguishes itself through specialized support for sparse architectures, particularly mixture-of-experts mode

MemGPT is a memory management framework and external memory layer for large language models. It functions as a platform for building stateful AI agents that maintain a persistent identity and continuous context across multiple sessions. The system enables agents to bypass fixed context window limitations by using a virtual context windowing approach. This allows models to manage their own memory through internal commands to search, update, and delete stored information within a hierarchical structure of short-term working context and long-term archival storage. The framework provides a local

Parallax is a distributed model serving framework that lets you build your own AI cluster anywhere

LangChain is an orchestration framework designed for building, managing, and deploying applications powered by large language models. It provides a unified integration layer that normalizes disparate model provider APIs into a consistent set of primitives, enabling developers to build complex, multi-step AI workflows that manage state, memory, and tool execution. The project distinguishes itself through a durable execution runtime that maintains persistent state across long-running processes by checkpointing progress to external storage. It models agent workflows as directed graphs, allowing

Llama.cpp is an inference engine designed for the local execution of text-based and multimodal language models on consumer hardware. It provides a core environment for running models that process both text and image inputs, utilizing hardware-accelerated backends to optimize performance across diverse CPU and GPU architectures. The project distinguishes itself by offering a lightweight HTTP server that adheres to standard API specifications, enabling chat completion, embeddings, and reranking services. It includes a suite of tools for model quantization and conversion, which reduces memory us

A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators.

Flowise is a low-code platform designed for building and deploying complex language model workflows through a visual, node-based interface. It functions as an orchestrator for autonomous multi-agent systems, allowing users to construct conversational pipelines by connecting language models, memory stores, and external tools on a drag-and-drop canvas. The platform distinguishes itself through its support for sophisticated agentic patterns, including supervisor-worker delegation and iterative reasoning strategies. Users can design directed acyclic graphs to manage conditional branching, state p

OptiLLM is an inference proxy and gateway router that directs prompts to specific language models based on cost, performance, and provider health. It functions as a middleware layer designed to optimize requests through intelligent routing, load balancing, and context management. The project provides specialized capabilities for data protection by anonymizing personally identifiable information before requests reach a model. It also acts as a reasoning orchestrator and tool integration layer, using inference-time loops and self-reflection to improve accuracy while connecting models to externa

llama.cpp is a high-performance C++ inference engine and runtime for executing large language models locally across various hardware architectures. It provides the core components for local model execution, including a dedicated model quantizer for compressing weights into the GGUF format and a system for generating text embeddings for semantic search. The project distinguishes itself through specialized memory and execution optimizations, such as block-wise weight quantization to reduce memory footprints and memory-mapped model loading. It supports structured text generation by using formal

Infinity is a distributed vector database and multimodal vector store designed to manage large-scale datasets for retrieval and similarity search. It serves as a backend for large language model applications and retrieval augmented generation pipelines by storing and retrieving dense vectors, sparse vectors, and full-text data. The system functions as a hybrid search engine, combining vector embeddings and full-text search with reranking algorithms to identify the most relevant documents. It supports multimodal data storage, allowing the maintenance of diverse data types including tensors, st

Install Stable Install Latest

OpenLLM is a framework for deploying, managing, and scaling open-source large language models

LiteLLM is a unified gateway and proxy server designed to centralize access to over one hundred language model providers. It provides a standardized API interface that abstracts vendor-specific schemas, allowing developers to interact with diverse models through a single, consistent format. By acting as a central traffic management layer, it enables organizations to route, secure, and govern model interactions across multiple deployments. The platform distinguishes itself through its policy-driven architecture, which uses configuration-based routing to manage traffic distribution, load balanc

This project is a web-based user interface for interacting with large language models, featuring streaming responses and persistent conversation history. It functions as an orchestration gateway that directs user prompts to specific language models and acts as a Model Context Protocol client to execute external tools and incorporate live data into conversations. The application includes a routing layer that analyzes input signals and tool requirements to dynamically direct messages to the most appropriate specialized model. It also provides customization settings for brand identity, allowing

lmdeploy is a high-performance inference engine and deployment framework for large language models and vision models. It functions as a multi-modal model server and compression toolkit designed to serve models with high throughput and low latency. The system enables the distribution of model services across multiple machines using request-based load balancing and tensor parallelism. It includes specialized tools for model quantization and compression to reduce the memory footprint of weights and caches. The framework covers broad capability areas including production deployment, distributed