llama.cpp is a high-performance C++ inference engine and runtime for executing large language models locally across various hardware architectures. It provides the core components for local model execution, including a dedicated model quantizer for compressing weights into the GGUF format and a system for generating text embeddings for semantic search. The project distinguishes itself through specialized memory and execution optimizations, such as block-wise weight quantization to reduce memory footprints and memory-mapped model loading. It supports structured text generation by using formal
Llama.cpp is an inference engine designed for the local execution of text-based and multimodal language models on consumer hardware. It provides a core environment for running models that process both text and image inputs, utilizing hardware-accelerated backends to optimize performance across diverse CPU and GPU architectures. The project distinguishes itself by offering a lightweight HTTP server that adheres to standard API specifications, enabling chat completion, embeddings, and reranking services. It includes a suite of tools for model quantization and conversion, which reduces memory us
Agentset MCP Server - Build RAG with Agentic superpowers
LangChain is an orchestration framework designed for building, managing, and deploying applications powered by large language models. It provides a unified integration layer that normalizes disparate model provider APIs into a consistent set of primitives, enabling developers to build complex, multi-step AI workflows that manage state, memory, and tool execution. The project distinguishes itself through a durable execution runtime that maintains persistent state across long-running processes by checkpointing progress to external storage. It models agent workflows as directed graphs, allowing
On-premises conversational RAG with configurable containers
dmayboroda/minima की मुख्य विशेषताएं हैं: AI & Machine Learning, Large Language Models, Knowledge and Memory, Knowledge Management।
dmayboroda/minima के ओपन-सोर्स विकल्पों में शामिल हैं: ggml-org/llama.cpp — Llama.cpp is an inference engine designed for the local execution of text-based and multimodal language models on… lm-sys/fastchat — FastChat is a training and serving platform for large language models that provides an integrated toolkit for… agentset-ai/mcp-server — Agentset MCP Server - Build RAG with Agentic superpowers. ggerganov/llama.cpp — llama.cpp is a high-performance C++ inference engine and runtime for executing large language models locally across… langchain-ai/langchain — LangChain is an orchestration framework designed for building, managing, and deploying applications powered by large… nomic-ai/gpt4all — GPT4All is a cross-platform runtime environment designed to execute large language models directly on local consumer…