Projects sharing features with Fauxpilot

The nexa-sdk is an on-device AI SDK and multimodal inference engine designed to run large language, vision, and audio models locally on mobile and desktop hardware. It functions as a local LLM runtime and NPU acceleration framework, enabling the execution of generative and discriminative models without reliance on cloud services. The project distinguishes itself through a dedicated NPU acceleration framework that optimizes model execution on Neural Processing Units to reduce latency and power consumption. It employs hardware-agnostic backend routing to dynamically distribute computations acro

LocalAI is a local generative AI platform and inference engine designed to host large language, vision, and audio models on private hardware. It functions as an API compatible gateway that mimics proprietary service endpoints, allowing existing third-party software to integrate with a self-hosted backend. The platform distinguishes itself as a distributed AI model orchestrator, capable of scaling inference across machine clusters using VRAM-aware routing and hardware coordination. It provides a unified interface for diverse open-source backends and supports self-hosted RAG infrastructure thro

ollama-python is a Python client for interacting with large language models. It provides an interface for sending prompts to receive text and chat completions, as well as a dedicated client for generating numerical vector embeddings from text. The project includes a wrapper that emulates the OpenAI API, allowing applications built for that standard to interact with local models. It also provides a non-blocking asynchronous client for executing concurrent requests. The library covers the full model lifecycle, including the ability to pull, create, list, and delete models within a local enviro

CoAI is an enterprise-grade, self-hostable AI gateway platform that unifies access to over 200 AI models from more than 35 providers through a single OpenAI-compatible API endpoint. It functions as a multi-tenant gateway, routing requests across providers with load balancing, automatic failover, and priority-based routing, while exposing standard OpenAI API endpoints for chat, image generation, model listing, and billing to enable seamless integration with existing tools and clients. The platform distinguishes itself through a comprehensive set of operational capabilities built around the gat

TTS-WebUI is a web interface and speech synthesis manager designed to convert written text into spoken audio files. It serves as a self-hosted audio AI suite that allows users to configure speech synthesis models, manage speaker profiles, and generate audio through a graphical dashboard. The system functions as both a visual manager and a generative audio API, providing standardized endpoints and OpenAI-compatible request formats for external applications to trigger synthesis programmatically. It includes a plugin-based extension system that allows new tools and models to be added via externa

mistral.rs is an inference engine for large language models that runs locally and exposes models behind OpenAI and Anthropic-compatible APIs. It serves as a multi-model serving platform, capable of loading several models in a single server process with per-request routing and on-demand loading and unloading. The engine supports multimodal inference, processing text alongside images, video, audio, and speech inputs, and includes a quantized model deployment runtime that reduces memory use and speeds up inference on consumer hardware. The project distinguishes itself through an agentic tool exe

This project is an API proxy that provides free and paid access to ChatGPT models through an OpenAI-compatible endpoint. It acts as a reverse proxy, routing requests to ChatGPT while maintaining full compatibility with OpenAI's SDK interface, allowing any application or tool that supports a custom base URL and API key to connect. The service offers a free tier that provides access to ChatGPT models for chat, image generation, and voice dialogue without requiring an official subscription, along with a paid tier that unlocks over 130 OpenAI models including GPT-4 with lower latency and reduced

FastDeploy is a high-performance deployment framework for large language models, vision models, and multimodal models. It provides the infrastructure to launch model services that process combined image, video, and text inputs, exposing these capabilities through a standardized, OpenAI-compatible API for chat and text completions. The project distinguishes itself through advanced inference pipeline engineering and GPU optimization. It employs speculative decoding, tensor parallelism, and a disaggregated execution model that separates prefill and decode phases across different hardware resourc

KServe is a Kubernetes-native platform for deploying and serving machine learning models as scalable inference services. It supports both generative AI models, including large language models, and traditional predictive models from frameworks such as TensorFlow, PyTorch, Scikit-Learn, XGBoost, and ONNX. The platform manages the full lifecycle of model deployments, including revision tracking, canary rollouts, A/B testing, and automatic rollbacks, and provides serverless scale-to-zero capabilities for cost-efficient resource management. KServe distinguishes itself through a standardized infere

Yao is an LLM agent framework and low-code web app builder designed for orchestrating autonomous AI agents. It provides a platform to design, deploy, and coordinate agents with specialized personas that can plan tasks, utilize external tools, and execute multi-stage pipelines. The project distinguishes itself through a Model Context Protocol server for connecting assistants to external binaries and HTTP services, and a gRPC remote execution engine that allows agents to manage remote servers and devices. It includes a model-agnostic provider bridge that supports dynamic switching between vario

MiniCPM is a collection of small language models designed for local, on-device deployment in resource-constrained environments. The project focuses on running dense Transformer models on consumer hardware, including GPUs, CPUs, and Apple Silicon, without requiring custom code forks. The project distinguishes itself through heavy optimization for edge hardware, utilizing quantized weight compression in GGUF and MLX formats to reduce memory overhead. It implements advanced inference techniques such as speculative sampling and radix-tree prefix caching to accelerate generation speed and throughp

This project is a multi-provider AI gateway and proxy server that intercepts and routes requests between AI clients and various large language model providers. It functions as an API protocol translator and model router, mapping incoming requests to specific upstream providers or local runners to provide a unified interface for multiple models. The system distinguishes itself by bridging chat platforms and command line interfaces, converting messages from chat services into managed command line sessions. It further optimizes traffic by executing certain web search and fetch requests locally a

GPT4All is a cross-platform runtime environment designed to execute large language models directly on local consumer hardware. By leveraging an optimized C++ inference backend, it enables private, offline AI interactions without requiring an internet connection or external cloud services. The project provides a comprehensive ecosystem for managing the entire model lifecycle, including discovery, downloading, and configuration of local weights. What distinguishes the platform is its integrated retrieval-augmented generation engine, which allows users to index local documents into semantic vect

Ktransformers is a comprehensive framework designed for the operation, fine-tuning, and serving of large language models. It functions as a heterogeneous inference engine and quantized execution runtime, enabling the deployment of massive models by distributing computational workloads across both CPU and GPU resources. This architecture allows users to bypass local memory constraints, making it possible to run and train models that exceed the capacity of a single device. The project distinguishes itself through specialized support for sparse architectures, particularly mixture-of-experts mode

Langroid is a multi-agent orchestration framework and tool integration suite designed for building complex AI applications. It serves as a multi-modal integration layer that connects diverse local and remote language models with an agentic retrieval-augmented generation system. The project distinguishes itself through a collaborative message-exchange paradigm, allowing specialized agents to delegate tasks hierarchically and coordinate via structured communication. It features an advanced state management system for conversational AI, including the ability to rewind and prune conversation hist

mcp-context-forge is a Model Context Protocol federation gateway that unifies diverse AI tool servers and APIs into a single consistent interface for discovery and execution. It acts as a centralized proxy that aggregates multiple servers and APIs, allowing AI agents to access and invoke a unified set of tools, prompts, and resources. The project distinguishes itself through a multi-protocol translation bridge that converts communication between standard I/O, SSE, gRPC, and REST to enable interoperability between disparate tool servers. It includes a comprehensive LLM evaluation framework for

Devika is an autonomous AI software engineering system designed to plan, write, and debug code from high-level natural language instructions. It functions as an agentic software engineer that decomposes complex objectives into actionable coding steps for autonomous execution. The system integrates cloud-based and self-hosted large language models through a provider-agnostic layer, allowing for multi-model reasoning and code completion. It distinguishes itself by combining these models with a sandboxed execution environment for running code across different operating systems and a web-browsing

Mastra is an orchestration framework designed for building, deploying, and managing autonomous AI agents and multi-agent systems. It provides a comprehensive suite of primitives for creating resilient AI applications, including durable workflow orchestration, event-driven agent loops, and semantic memory management. By integrating these core components, the platform enables developers to build complex, multi-step processes that can reason about goals and execute tasks without manual intervention. The framework distinguishes itself through its focus on observability and secure, isolated execut

Atmosphere is a Java-based framework for building and coordinating AI agents. It provides a real-time transport layer for streaming data via WebSockets, SSE, gRPC, and WebTransport, alongside a multi-agent orchestration framework for managing agent fleets through sequential, parallel, and graph-based execution workflows. The project features a durable workflow engine that persists agent state as snapshots, allowing long-running tasks to survive system restarts and incorporate human-in-the-loop approvals. It also implements Model Context Protocol servers to expose tools, resources, and prompt

MNN is a high-performance inference engine and framework designed for on-device machine learning. It provides a comprehensive environment for executing, optimizing, and deploying neural network models directly on mobile and resource-constrained edge devices. The framework distinguishes itself through a robust model optimization toolkit that supports quantization, compression, and structural graph manipulation to minimize memory footprint and maximize execution speed. It features a modular architecture that abstracts hardware-specific backends, allowing models to run efficiently across diverse

Khoj is a self-hosted artificial intelligence platform designed for personal knowledge management and semantic information retrieval. It functions as a private assistant that indexes your local documents, notes, and external workspaces, allowing you to interact with your data through natural language queries and conversational chat. By maintaining a local-first architecture, the system ensures that your information remains under your control while providing context-aware responses grounded in your personal knowledge base. The platform distinguishes itself through a modular, cross-platform int

Kilocode is an autonomous engineering platform designed to orchestrate AI agents for complex software development tasks. It functions as a comprehensive system for automating coding, testing, and repository management by integrating directly with your codebase and terminal. The platform provides a unified gateway for model orchestration, allowing for the management of agentic workflows, event-driven automation, and persistent session state across distributed development environments. The platform distinguishes itself through its federated task management and policy-based access control, which

BAML is a prompt engineering framework and LLM client generator that defines AI prompts as type-safe functions. It serves as a structured data extraction tool and workflow orchestrator, transforming unstructured model responses into strongly typed objects using a custom schema language and alignment algorithms. The project distinguishes itself by using a compiler to generate language-specific boilerplate code for API communication and output parsing. It features a dedicated environment for designing complex prompt templates with conditional logic and reusable snippets, and employs genetic alg

This project provides a dockerized AI workflow stack and orchestration templates for deploying a self-hosted AI environment. It establishes a localized infrastructure for building autonomous agents and model chains that process private data on-premises without external cloud dependencies. The environment is designed to support autonomous agent development, allowing models to dynamically select tools, execute shell commands, and interact with local file systems. It includes integrated vector database support to enable retrieval augmented generation and private document analysis. The stack cov

GGML is a machine learning tensor library and neural network engine written in C. It functions as a compute-focused runtime designed to execute transformer-based models and perform complex mathematical operations on multi-dimensional arrays directly on local consumer hardware. The library distinguishes itself by enabling local inference for large language models and edge machine learning deployment without reliance on external cloud infrastructure. It achieves this through a tensor-based computation graph that organizes operations for efficient execution and memory management, alongside stati

Llama-GPT is a self-hosted generative AI model runner that provides a private web interface for interacting with large language models. By executing these models directly on local hardware, it ensures that all intelligent assistance remains offline and independent of external cloud service providers. The project functions as a private assistant that maintains complete data ownership by storing all application state and model interactions on local storage volumes. It is designed to operate within a broader self-hosted computing environment, allowing users to maintain control over their persona