awesome-repositories.com
Blog
MCP
awesome-repositories.com

Discover the best open-source repositories with AI-powered search.

ExploreCurated searchesOpen-source alternativesSelf-hosted softwareBlogSitemap
ProjectMCP serverAboutHow we rankPress
LegalPrivacyTerms
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
Back to microsoft/foundry-local

Projects sharing features with Foundry Local

30 open-source projects similar to microsoft/foundry-local, ranked by shared indexed features. Tags may describe platforms or build tools rather than the same primary purpose. Check each project’s use case, license, and deployment requirements before treating it as a replacement.

  • soniqo/speech-swiftsoniqo avatar

    soniqo/speech-swift

    896View on GitHub↗

    This project is a comprehensive toolkit for on-device speech recognition, synthesis, and audio processing, specifically engineered for Apple Silicon. It provides a framework for building real-time, full-duplex voice agents that operate entirely offline, leveraging native hardware acceleration to maintain performance and privacy. By utilizing optimized machine learning models, the library enables local execution of complex audio tasks without reliance on external cloud services. The library distinguishes itself through its specialized focus on local, high-performance voice interaction. It incl

    Swiftapple-siliconasrcoreml
    View on GitHub↗896
  • microsoft/skill-recordermicrosoft avatar

    microsoft/skill-recorder

    5,710View on GitHub↗

    Skill Recorder is a desktop app that records an on-screen work session and turns it into a reusable skill or scheduled automation for AI agents. It captures clicks, app and window switches, visited pages, clipboard previews, terminal activity and spoken narration in the background, controlled by a global shortcut and a floating bar. It can also import an existing MP4 or WebM demonstration instead of recording live. Everything is recorded and stored locally until the user chooses to analyze it. Analysis uses the GitHub Copilot SDK, with sign-in through the browser via enterprise single sign-on

    TypeScriptagent-skillsai-agentsautomation
    View on GitHub↗5,710
  • google-ai-edge/gallerygoogle-ai-edge avatar

    google-ai-edge/gallery

    15,162View on GitHub↗

    This project is a development framework for building edge-based AI agents that perform multimodal inference and system-level automation directly on mobile devices. By prioritizing local-first execution, the platform ensures data privacy and offline functionality, allowing developers to run large language models on hardware without requiring external server connectivity. The framework distinguishes itself through an integrated orchestration layer that connects language models to custom tools, scripts, and native device intents. It provides a structured registry for mapping natural language ins

    Kotlin
    View on GitHub↗15,162

AI search

Explore more awesome repositories

Describe what you need in plain English — the AI ranks thousands of curated open-source projects by relevance.

Find more with AI search
  • cocktailpeanut/dalaicocktailpeanut avatar

    cocktailpeanut/dalai

    12,920View on GitHub↗

    The simplest way to run LLaMA on your local machine

    CSSaillamallm
    View on GitHub↗12,920
  • fmhy/fmhyfmhy avatar

    fmhy/FMHY

    13,150View on GitHub↗

    FMHY is a community-driven index designed to organize and distribute decentralized digital content through standardized metadata and protocol-agnostic linking. It functions as a resilient, distributed map of internet resources, providing a structured directory that facilitates the discovery of media, software, and educational tools without reliance on centralized control. The project distinguishes itself by maintaining a massive, human-verified repository of external links that span diverse digital ecosystems, including peer-to-peer networks, Usenet, and direct download servers. By utilizing

    freemediaheckyeah
    View on GitHub↗13,150
  • bentoml/bentomlbentoml avatar

    bentoml/BentoML

    8,934View on GitHub↗

    BentoML is a machine learning model serving framework and GPU-accelerated inference server designed to package, deploy, and scale AI models as production-ready REST APIs. It functions as an AI model lifecycle manager and an inference graph orchestrator, enabling the chaining of multiple models and custom logic into complex pipelines for advanced task sequences. The framework distinguishes itself through a dynamic batching engine that optimizes GPU throughput and an artifact-based packaging system that bundles model weights and dependencies into immutable archives for consistent deployment. It

    Pythonai-inferencedeep-learninggenerative-ai
    View on GitHub↗8,934
  • google-ai-edge/litert-lmgoogle-ai-edge avatar

    google-ai-edge/LiteRT-LM

    6,840View on GitHub↗

    LiteRT-LM is a high-performance inference framework designed to execute large language models locally on mobile, desktop, and IoT hardware. It serves as an on-device model runtime that utilizes CPU, GPU, and NPU acceleration to provide low-latency processing. The framework is distinguished by its ability to process text, vision, and audio inputs through a single multi-modal inference engine. It features a local HTTP server that emulates OpenAI-compatible API endpoints and a WebGPU-based runtime for executing models directly within a web browser. To ensure output reliability, it includes a con

    C++
    View on GitHub↗6,840
  • cactus-compute/cactuscactus-compute avatar

    cactus-compute/cactus

    5,363View on GitHub↗

    Cactus is an on-device AI inference engine designed for executing large language models, vision models, and speech-to-text systems on mobile and wearable hardware. It provides a programmable tensor computation graph for defining sequences of matrix operations and activation functions, alongside a local retrieval augmented generation framework that grounds model responses using local text files. The project features a multiplatform SDK with language bindings for integrating AI capabilities into mobile applications and a model conversion system that transforms external model formats for optimiz

    C++aiandroidarm
    View on GitHub↗5,363
  • mlc-ai/web-llmmlc-ai avatar

    mlc-ai/web-llm

    18,208View on GitHub↗

    WebLLM is a library for executing large language models directly within web browsers. It provides a framework for building conversational artificial intelligence applications that perform inference locally, ensuring user data privacy by eliminating the need for external server dependencies. The project distinguishes itself by leveraging browser-native graphics APIs to perform intensive machine learning computations on the client side. It maintains application responsiveness by offloading heavy model tasks to background threads and ensures continuous operation through service workers that func

    TypeScriptchatgptdeep-learninglanguage-model
    View on GitHub↗18,208
  • vladmandic/sdnextvladmandic avatar

    vladmandic/sdnext

    7,139View on GitHub↗

    SD.Next is an all-in-one web interface and multi-backend inference engine for generating, editing, and processing images and videos using diffusion models. It functions as a comprehensive tool for diffusion model management and an automated image processing pipeline for bulk operations. The project is distinguished by its hardware-backend abstraction layer, which provides automatic detection and acceleration for NVIDIA CUDA, AMD ROCm, Intel OpenVINO, and DirectML. It features a headless generative API and a programmatic command interface, allowing users to trigger tasks via REST API or CLI wi

    Pythonai-artcaptiondiffusers
    View on GitHub↗7,139
  • antirez/ds4antirez avatar

    antirez/ds4

    24,264View on GitHub↗

    ds4 is a local inference engine for DeepSeek models that includes a distributed runtime for splitting transformer layers across networked computers. It functions as a reasoning controller with a local weight streamer and an API server that streams chat completions via industry standard endpoints. The system employs a memory management model that loads model experts from disk on demand to execute models that exceed available system RAM. It provides controls for reasoning effort and model behavior steering, allowing the modification of response characteristics through activation directions. Th

    C
    View on GitHub↗24,264
  • openvinotoolkit/openvinoopenvinotoolkit avatar

    openvinotoolkit/openvino

    11,128View on GitHub↗

    OpenVINO is an AI inference engine and model serving platform designed to execute optimized deep learning models across CPUs, GPUs, and NPUs through a unified API. It includes a model optimization toolkit for converting, quantizing, and compressing models from various frameworks, alongside a specialized generative AI runtime for large language models. The project distinguishes itself through a plugin-based hardware acceleration layer that maps neural network operations to vendor-specific drivers. It features advanced execution mechanisms such as continuous batching, speculative decoding, and

    C++aicomputer-visiondeep-learning
    View on GitHub↗11,128
  • argmaxinc/whisperkitargmaxinc avatar

    argmaxinc/WhisperKit

    5,639View on GitHub↗
    Swiftinferenceiosmacos
    View on GitHub↗5,639
  • k2-fsa/sherpa-ncnnk2-fsa avatar

    k2-fsa/sherpa-ncnn

    1,743View on GitHub↗

    Sherpa-ncnn is an edge-based speech recognition and synthesis engine designed to run neural network models locally on mobile, embedded, and desktop hardware. It provides a cross-platform framework for offline speech-to-text transcription and text-to-speech synthesis, ensuring that all audio processing occurs on-device without requiring an internet connection or external cloud services. The project distinguishes itself through its use of the ncnn inference engine, which is optimized for low-latency execution on resource-constrained devices. It incorporates on-device model quantization to reduc

    C++asrccpp
    View on GitHub↗1,743
  • humanitas-labs/quillhumanitas-labs avatar

    humanitas-labs/quill

    3,960View on GitHub↗

    Quill is a native Swift menu-bar app for macOS that records meetings and calls. It captures the microphone and a chosen application's audio as two independent tracks, so the user's own voice stays separate from remote participants. Each recording is saved into its own timestamped session folder as per-track WAV files with a metadata file describing timing. After recording stops, it transcribes the audio on the user's machine with a downloadable speech model, so no audio or transcript leaves the device. Several things set it apart. Capture can be limited to a single selected application and it

    Swift
    View on GitHub↗3,960
  • crmne/ruby_llmcrmne avatar

    crmne/ruby_llm

    4,498View on GitHub↗

    ruby_llm is an LLM integration framework and AI agent orchestrator designed to connect applications to multiple large language model providers through a unified interface. It serves as a toolkit for building autonomous assistants with custom personas, managing structured output via JSON schemas, and implementing vector embedding engines for semantic search. The project distinguishes itself as an observability suite and multimodal toolkit. It provides specialized capabilities for tracking token usage, calculating model costs, and tracing workflows via OpenTelemetry, while supporting the proces

    Rubyaianthropicchatgpt
    View on GitHub↗4,498
  • jundot/omlxjundot avatar

    jundot/omlx

    17,112View on GitHub↗

    omlx is a local inference server designed to run large language models, vision models, and embedding models on Apple Silicon. It provides a private alternative to industry-standard AI endpoints by hosting a local API gateway that mirrors OpenAI and Anthropic specifications. The system distinguishes itself through specialized hardware optimizations, including continuous batching for high throughput and a tiered caching system that offloads memory blocks to SSD. It also functions as a Model Context Protocol host, enabling the integration of local models with external tools, agents, and structur

    Python
    View on GitHub↗17,112
  • microsoft/onnxruntimemicrosoft avatar

    microsoft/onnxruntime

    19,347View on GitHub↗

    This project is a cross-platform machine learning inference engine designed to execute pre-trained models across diverse operating systems and hardware environments. It functions as a standardized execution framework that manages the entire lifecycle of model inference, from loading and graph optimization to hardware-accelerated execution and generative sequence management. The runtime distinguishes itself through a highly modular architecture that decouples model logic from hardware-specific kernels. By utilizing an execution provider abstraction, it enables developers to offload computation

    C++ai-frameworkdeep-learninghardware-acceleration
    View on GitHub↗19,347
  • jamsch/expo-speech-recognitionjamsch avatar

    jamsch/expo-speech-recognition

    541View on GitHub↗

    Expo Speech Recognition is a cross-platform mobile module that converts live microphone audio and pre-recorded files into text using native speech engines. It provides offline speech recognition capabilities by downloading and verifying local speech models to enable on-device processing without an active network connection. The library includes session lifecycle management to start, stop, or abort recording, alongside real-time spoken language detection with confidence scoring. It emits volume change events for metering interfaces, handles audio session configuration and routing, and persist

    TypeScriptexporeact-nativespeech-recognition
    View on GitHub↗541
  • ggml-org/llama.cppggml-org avatar

    ggml-org/llama.cpp

    133,666View on GitHub↗

    Llama.cpp is an inference engine designed for the local execution of text-based and multimodal language models on consumer hardware. It provides a core environment for running models that process both text and image inputs, utilizing hardware-accelerated backends to optimize performance across diverse CPU and GPU architectures. The project distinguishes itself by offering a lightweight HTTP server that adheres to standard API specifications, enabling chat completion, embeddings, and reranking services. It includes a suite of tools for model quantization and conversion, which reduces memory us

    C++ggml
    View on GitHub↗133,666
  • google-ai-edge/litertgoogle-ai-edge avatar

    google-ai-edge/LiteRT

    3,577View on GitHub↗

    LiteRT is a runtime and API for executing machine learning and generative AI models on mobile, desktop, and IoT hardware. It consists of an inference engine and a specialized environment for running quantized large language and diffusion models locally on edge hardware. The system includes an ahead-of-time model compiler that translates models into hardware-specific bytecode to reduce startup latency and memory overhead. It provides a unified interface for Neural Processing Units with automatic fallback routing to CPUs or GPUs when specific subgraph support is unavailable. An edge model conve

    C++
    View on GitHub↗3,577
  • exo-explore/exoexo-explore avatar

    exo-explore/exo

    48,237View on GitHub↗

    Exo is a distributed inference engine designed to run machine learning models across local hardware. It functions as a network orchestration layer that automatically discovers available devices to form a unified computing cluster, allowing users to scale artificial intelligence workloads by distributing computational tasks across multiple machines. The platform distinguishes itself through its ability to manage the entire lifecycle of local models while providing a standardized gateway for external applications. By translating local model outputs into industry-standard formats, it enables exi

    Python
    View on GitHub↗48,237
  • linuxhsj/openclaw-zero-tokenlinuxhsj avatar

    linuxhsj/openclaw-zero-token

    5,213View on GitHub↗

    OpenClaw Zero Token is a self-hosted gateway and assistant platform that lets people chat with browser-based AI services such as ChatGPT, Claude, Gemini, DeepSeek, Kimi and Grok without paying for API tokens. It captures a login from a debuggable Chrome browser through the Chrome DevTools Protocol, then reuses that session's cookies and headers for later requests. A single OpenAI-compatible chat completions endpoint, secured by a bearer token, routes each request to whichever web model is configured, so outside programs can call those models like a standard API. Because web chat models have n

    TypeScript
    View on GitHub↗5,213
  • maziyarpanahi/openmedmaziyarpanahi avatar

    maziyarpanahi/openmed

    5,785View on GitHub↗

    OpenMed is a local-first healthcare AI toolkit for clinical text. It finds diseases, drugs, anatomy, genes and personal identifiers in medical notes, then masks, replaces, hashes or date-shifts the identifiers under HIPAA, GDPR and research-style policy profiles. It also turns extracted entities into terminology-coded FHIR resources and OMOP records. A curated registry of specialized medical entity-recognition models, selectable by name, sits underneath all of this. Its de-identification work covers dozens of languages and scripts. It uses locale-aware fake replacements, checksum-valid nation

    Pythonandroidclinical-nlphealthcare
    View on GitHub↗5,785
  • menloresearch/janmenloresearch avatar

    menloresearch/jan

    43,052View on GitHub↗

    Jan is a local language model desktop application and AI assistant orchestrator. It provides a unified interface for interacting with both resident models and remote cloud AI providers. The project functions as a host for the Model Context Protocol, connecting AI models to external tools and data sources. It also operates as an OpenAI compatible API server, exposing local models through a standardized server endpoint for other applications to query. The system supports the creation of specialized AI personas with custom instructions and allows for the management of hybrid model environments,

    TypeScript
    View on GitHub↗43,052
  • llmware-ai/llmwarellmware-ai avatar

    llmware-ai/llmware

    14,821View on GitHub↗

    llmware is a Python framework for AI agent orchestration and model management, designed to coordinate multi-model workflows and autonomous agents. It provides a unified model catalog and standardized interface to execute specialized language models for complex research, analysis, and structured data generation. The project distinguishes itself through its heavy emphasis on local execution and quantized inference, allowing models to run on private infrastructure using CPU, GPU, and NPU acceleration via runtimes like ONNX and OpenVino. It features a specialized ability to translate natural lang

    Python
    View on GitHub↗14,821
  • turboderp-org/exllamav2turboderp-org avatar

    turboderp-org/exllamav2

    4,651View on GitHub↗

    exllamav2 is a high-performance inference engine and framework for executing large language models locally on consumer-class GPUs. It provides a complete system for local model deployment, including a specialized inference engine and tools for model quantization. The project features a multi-GPU inference framework that distributes workloads across multiple graphics cards to run models that exceed the memory capacity of a single device. It includes a GPU model quantizer capable of converting models into mixed-precision formats between 2 and 8 bits to balance memory usage and accuracy. The en

    Python
    View on GitHub↗4,651
  • apache/predictionioapache avatar

    apache/predictionio

    12,522View on GitHub↗

    PredictionIO is a machine learning server designed for the deployment of predictive models to transform raw data into actionable predictions. It manages the full lifecycle of machine learning operations, from ingesting event data via APIs to hosting production-ready predictive services for real-time inference. The system supports distributed model training by spreading computational workloads across a cluster of nodes to increase processing speed. It enables the implementation of custom prediction engines using programming languages or the application of pre-built model templates for common t

    Scala
    View on GitHub↗12,522
  • antimatter15/alpaca.cppantimatter15 avatar

    antimatter15/alpaca.cpp

    10,138View on GitHub↗

    alpaca.cpp is a high-performance local inference engine implemented in C++ for executing instruction-tuned large language models. It serves as a quantized model runtime designed to load and run model tensors on local hardware with minimal dependencies, removing the requirement for a full Python environment. The project focuses on on-device text generation and the deployment of private AI chatbots. It utilizes model weight quantization to reduce memory requirements and increase inference speed on consumer-grade devices. The system covers hardware-optimized model execution through thread-pool

    C
    View on GitHub↗10,138
  • runanywhereai/runanywhere-sdksRunanywhereAI avatar

    RunanywhereAI/runanywhere-sdks

    8,781View on GitHub↗

    This project is an on-device AI SDK providing a framework for running large language models, vision models, and speech models locally. It serves as an orchestration layer for local LLM execution, ensuring data privacy and offline availability by utilizing hardware acceleration on the device. The SDK is distinguished by its comprehensive voice and multimodal capabilities, including a coordinated voice pipeline for activity detection, speech-to-text, and text-to-speech synthesis. It also provides a dedicated implementation kit for local retrieval-augmented generation and tools for processing co

    C++androidapple-intelligencecpp
    View on GitHub↗8,781