ExecuTorch is a lightweight C++ runtime for deploying PyTorch models on mobile, embedded, and edge hardware. It provides an ahead-of-time compilation pipeline that exports, quantizes, and lowers model graphs into compact serialized programs, then executes them through a minimal runtime with hardware acceleration and on-device large language model inference capabilities. The project distinguishes itself through a hardware accelerator delegate system that partitions model subgraphs and offloads computation to specialized backends including NPUs, GPUs, and DSPs from Apple, Arm, Intel, MediaTek,
Mistral Inference is a library for running Mistral large language models on a GPU, generating text from prompts with token streaming. It loads pretrained model weights from local disk or a remote registry into GPU memory, then produces output tokens one by one for real-time display in interactive applications. The library supports multimodal prompts that accept image URLs alongside text, enabling visual description and reasoning. It includes content safety guardrails that scan generated text against predefined policies to block or flag policy violations. For structured interactions, it provid
quant-wiki is a comprehensive knowledge base and structured reference for quantitative finance, financial engineering, and algorithmic trading. It serves as a centralized library of documentation covering mathematical models, financial instruments, and systematic trading strategies. The project integrates AI-driven capabilities through a modular retrieval-augmented generation framework that extracts structured data from research papers and news. It features a multi-agent workflow engine designed to discover and validate predictive alpha factors, alongside tools for local large language model
omlx is a local inference server designed to run large language models, vision models, and embedding models on Apple Silicon. It provides a private alternative to industry-standard AI endpoints by hosting a local API gateway that mirrors OpenAI and Anthropic specifications. The system distinguishes itself through specialized hardware optimizations, including continuous batching for high throughput and a tiered caching system that offloads memory blocks to SSD. It also functions as a Model Context Protocol host, enabling the integration of local models with external tools, agents, and structur