# Text Embedding Generation Libraries

> Search results for `generate text embeddings for semantic search` on awesome-repositories.com. 110 total matches; showing the first 50.

Explore on the web: https://awesome-repositories.com/q/generate-text-embeddings-for-semantic-search

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [this search on awesome-repositories.com](https://awesome-repositories.com/q/generate-text-embeddings-for-semantic-search).**

## Results

- [modular/modular](https://awesome-repositories.com/repository/modular-modular.md) (26,357 ⭐) — Modular is a unified machine learning development platform designed for building, compiling, and deploying high-performance neural network models. It provides a comprehensive execution engine that supports both local and production-grade inference, enabling developers to manage the entire model lifecycle from initial architecture definition to scalable, containerized service deployment.

The platform distinguishes itself through a hardware-agnostic runtime that abstracts diverse silicon architectures, allowing models to execute efficiently across varied compute environments. It includes a spec
- [huggingface/text-generation-inference](https://awesome-repositories.com/repository/huggingface-text-generation-inference.md) (10,775 ⭐) — Text Generation Inference is a production-ready engine designed for the deployment and serving of large language models. It functions as a containerized runtime environment that manages model execution, scales across distributed hardware, and provides high-performance inference capabilities for demanding production environments.

The project distinguishes itself through advanced optimization techniques, including continuous batching to maximize hardware utilization and tensor parallelism to shard large models across multiple accelerator cards. It supports efficient inference through custom com
- [microsoft/generative-ai-for-beginners](https://awesome-repositories.com/repository/microsoft-generative-ai-for-beginners.md) (112,045 ⭐) — This project is a comprehensive, open-source educational curriculum designed to guide developers through the mastery of generative artificial intelligence. It provides a structured learning path that covers foundational concepts, prompt engineering, and the practical application of large language models. The repository serves as a central hub for skill acquisition, offering sequential modules that progress from basic model mechanics to advanced architectural patterns.

The curriculum distinguishes itself by focusing on the end-to-end lifecycle of intelligent software, including the implementat
- [mastra-ai/mastra](https://awesome-repositories.com/repository/mastra-ai-mastra.md) (21,221 ⭐) — Mastra is an orchestration framework designed for building, deploying, and managing autonomous AI agents and multi-agent systems. It provides a comprehensive suite of primitives for creating resilient AI applications, including durable workflow orchestration, event-driven agent loops, and semantic memory management. By integrating these core components, the platform enables developers to build complex, multi-step processes that can reason about goals and execute tasks without manual intervention.

The framework distinguishes itself through its focus on observability and secure, isolated execut
- [camel-ai/camel](https://awesome-repositories.com/repository/camel-ai-camel.md) (17,253 ⭐) — This project is a comprehensive framework for building and managing autonomous agent systems. It provides a unified architecture for orchestrating multi-agent societies, where specialized agents collaborate through roleplay to decompose and solve complex tasks. The system integrates language models with external environments, enabling agents to perform real-world actions through a standardized tool-calling abstraction layer.

The framework distinguishes itself through its focus on iterative reasoning and data reliability. It employs automated feedback loops to refine agent outputs and self-eva
- [google-gemini/cookbook](https://awesome-repositories.com/repository/google-gemini-cookbook.md) (17,418 ⭐) — The Gemini Cookbook is a comprehensive collection of implementation patterns, code samples, and development guides designed for building applications with Google Gemini models. It serves as a central resource for developers to integrate multimodal generative artificial intelligence into their software, providing the necessary frameworks to manage model interactions, stateful workflows, and structured data extraction.

The repository distinguishes itself by offering specialized toolkits for autonomous agent orchestration, enabling the construction of agents that can execute code, browse the web
- [insforge/insforge](https://awesome-repositories.com/repository/insforge-insforge.md) (11,794 ⭐) — InsForge is a backend-as-a-service platform that provides an integrated suite of tools for managing relational databases, identity provision, object storage, and serverless compute. It functions as an open-source identity provider and a PostgreSQL database manager featuring integrated vector storage and row-level security.

The platform serves as an LLM orchestration gateway, offering a unified endpoint to route requests across various AI providers through an OpenAI-compatible interface. It enables AI-driven application generation and connects AI agents to backend resources using a standardize
- [cinnamon/kotaemon](https://awesome-repositories.com/repository/cinnamon-kotaemon.md) (25,139 ⭐) — Kotaemon is an orchestration framework designed for building modular, agentic workflows that integrate document processing, retrieval-augmented generation, and multi-step reasoning. It provides a comprehensive platform for developing document-based question answering systems, allowing users to chain language models, prompt templates, and external tools into complex, automated pipelines.

The system distinguishes itself through a highly modular architecture that emphasizes component-based composition and schema-driven data exchange. It supports autonomous agents capable of decomposing complex q
- [berriai/litellm](https://awesome-repositories.com/repository/berriai-litellm.md) (50,579 ⭐) — LiteLLM is a unified gateway and proxy server designed to centralize access to over one hundred language model providers. It provides a standardized API interface that abstracts vendor-specific schemas, allowing developers to interact with diverse models through a single, consistent format. By acting as a central traffic management layer, it enables organizations to route, secure, and govern model interactions across multiple deployments.

The platform distinguishes itself through its policy-driven architecture, which uses configuration-based routing to manage traffic distribution, load balanc
- [agentscope-ai/agentscope](https://awesome-repositories.com/repository/agentscope-ai-agentscope.md) (26,895 ⭐) — Agentscope is a comprehensive toolkit for developing and orchestrating autonomous multi-agent systems. It provides a unified framework for building agents that can reason, execute tools, and manage memory, enabling the creation of complex, collaborative workflows where multiple specialized agents interact to solve multi-step objectives.

The platform distinguishes itself through a robust orchestration engine that supports both sequential and concurrent agent pipelines. It utilizes a centralized event bus for real-time telemetry, allowing developers to track agent reasoning, tool usage, and sys
- [ryankiros/visual-semantic-embedding](https://awesome-repositories.com/repository/ryankiros-visual-semantic-embedding.md) (427 ⭐) — Code for the image-sentence ranking methods from "Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models" (Kiros, Salakhutdinov, Zemel. 2014).
- [zai-org/chatglm3](https://awesome-repositories.com/repository/zai-org-chatglm3.md) (13,764 ⭐) — ChatGLM3 is a comprehensive framework for deploying, fine-tuning, and serving large language models. It functions as a high-performance inference engine designed to support conversational AI, enabling developers to build interactive agents capable of multi-turn dialogue, autonomous code execution, and structured tool invocation.

The project distinguishes itself through its focus on hardware-agnostic deployment and resource optimization. It supports distributed model parallelism across multiple graphics cards, paged key-value caching for concurrent request processing, and weight quantization t
- [blakeblackshear/frigate](https://awesome-repositories.com/repository/blakeblackshear-frigate.md) (33,778 ⭐) — Frigate is a self-hosted network video recorder that functions as a private, local AI-powered vision engine. It manages video streams by performing real-time object detection, tracking, and classification directly on local hardware, ensuring that security monitoring and activity recording remain independent of cloud services.

The system distinguishes itself through a modular, hardware-accelerated video pipeline that offloads intensive decoding and machine learning inference to dedicated GPUs, NPUs, or specialized accelerators like Coral TPUs and Hailo modules. It utilizes state-based object t
- [huggingface/text-embeddings-inference](https://awesome-repositories.com/repository/huggingface-text-embeddings-inference.md) (4,871 ⭐) — A blazing fast inference solution for text embeddings models
- [openvinotoolkit/openvino](https://awesome-repositories.com/repository/openvinotoolkit-openvino.md) (10,414 ⭐) — OpenVINO is an AI inference engine and model serving platform designed to execute optimized deep learning models across CPUs, GPUs, and NPUs through a unified API. It includes a model optimization toolkit for converting, quantizing, and compressing models from various frameworks, alongside a specialized generative AI runtime for large language models.

The project distinguishes itself through a plugin-based hardware acceleration layer that maps neural network operations to vendor-specific drivers. It features advanced execution mechanisms such as continuous batching, speculative decoding, and
- [meilisearch/meilisearch](https://awesome-repositories.com/repository/meilisearch-meilisearch.md) (58,118 ⭐) — Meilisearch is a Rust-based search engine providing typo-tolerant full-text and vector-based semantic search with real-time conversational capabilities.
- [patricktrainer/duckdb-embedding-search](https://awesome-repositories.com/repository/patricktrainer-duckdb-embedding-search.md) (150 ⭐) — This repository contains a Python application that utilizes DuckDB as a backend to store and retrieve embedding vectors. The novel use of DuckDB allows for efficient similarity searches among large datasets. In this example, we've loaded comments from Hacker News and implemented functionality to…
- [flagopen/flagembedding](https://awesome-repositories.com/repository/flagopen-flagembedding.md) (11,833 ⭐) — FlagEmbedding is a comprehensive toolkit designed for training, benchmarking, and deploying embedding models, retrieval systems, and augmented generation pipelines. It provides the necessary infrastructure to transform text into high-dimensional vector representations and organize them into searchable structures for semantic search applications.

The framework distinguishes itself through specialized capabilities for fine-tuning pre-trained embedding and reranking models on domain-specific datasets. By allowing users to adapt models to unique vocabularies and specialized retrieval tasks, it en
- [oobabooga/text-generation-webui](https://awesome-repositories.com/repository/oobabooga-text-generation-webui.md) (47,323 ⭐) — This project is a comprehensive platform for hosting and interacting with large language models directly on local hardware. It provides a web-based graphical interface that allows users to manage model loading, configure generation parameters, and execute text or chat interactions entirely offline. By running models locally, the software ensures complete data privacy and eliminates reliance on external cloud services for generative tasks.

Beyond basic inference, the platform functions as a versatile workbench for generative AI development. It includes an integrated pipeline for fine-tuning mo
- [ggerganov/llama.cpp](https://awesome-repositories.com/repository/ggerganov-llama-cpp.md) (116,912 ⭐) — llama.cpp is a high-performance C++ inference engine and runtime for executing large language models locally across various hardware architectures. It provides the core components for local model execution, including a dedicated model quantizer for compressing weights into the GGUF format and a system for generating text embeddings for semantic search.

The project distinguishes itself through specialized memory and execution optimizations, such as block-wise weight quantization to reduce memory footprints and memory-mapped model loading. It supports structured text generation by using formal
- [milvus-io/milvus](https://awesome-repositories.com/repository/milvus-io-milvus.md) (44,804 ⭐) — Milvus is a specialized vector database engine designed for the indexing, management, and high-speed similarity retrieval of high-dimensional vector embeddings. It functions as a similarity search engine capable of identifying nearest neighbors within large-scale vector spaces, supporting the storage and retrieval of billions of data points while maintaining consistent performance.

The system utilizes a distributed architecture that decouples storage, query, and coordination into independent services, allowing for horizontal scaling across clusters. It employs a global indexing mechanism that
- [krisk/fuse](https://awesome-repositories.com/repository/krisk-fuse.md) (20,347 ⭐) — Fuse is a JavaScript fuzzy search library and client-side search engine designed to index and query JSON data. It provides utilities for approximate string matching and ranking results by relevance, allowing applications to perform fast filtering and searching of datasets without a dedicated backend.

The library distinguishes itself through a token-based search implementation that supports word-order independence and relevance weighting. It utilizes edit-distance scoring to handle typos and insertions, and employs a system of field weighting to prioritize matches in high-value data keys.

The
- [github/semantic](https://awesome-repositories.com/repository/github-semantic.md) (9,041 ⭐) — semantic is a Haskell library and command line tool for parsing, analyzing, and comparing source code.
- [mudler/localai](https://awesome-repositories.com/repository/mudler-localai.md) (46,889 ⭐) — LocalAI is a self-hosted inference server that enables the execution of machine learning models directly on local hardware. By providing a unified interface for text, image, and audio processing, it allows users to maintain full control over data privacy and infrastructure costs while eliminating dependencies on external network services.

The platform functions as an API gateway that mimics standard cloud-based artificial intelligence interfaces, allowing existing applications to integrate local models as drop-in replacements. It utilizes a container-based architecture to package runtimes and
- [embedded-graphics/embedded-graphics](https://awesome-repositories.com/repository/embedded-graphics-embedded-graphics.md) (1,295 ⭐) — A no_std graphics library for embedded applications
- [microsoft/unilm](https://awesome-repositories.com/repository/microsoft-unilm.md) (22,030 ⭐) — This project is a comprehensive framework and toolkit for developing, optimizing, and deploying transformer-based models across multimodal, document intelligence, and natural language processing tasks. It provides a unified neural architecture that processes text, vision, audio, and document layout data through a shared set of weights, enabling researchers and developers to build foundational models that align cross-modal representations.

The platform distinguishes itself through advanced training and inference strategies designed for large-scale deep learning. It incorporates specialized mec
- [kilo-org/kilocode](https://awesome-repositories.com/repository/kilo-org-kilocode.md) (15,616 ⭐) — Kilocode is an autonomous engineering platform designed to orchestrate AI agents for complex software development tasks. It functions as a comprehensive system for automating coding, testing, and repository management by integrating directly with your codebase and terminal. The platform provides a unified gateway for model orchestration, allowing for the management of agentic workflows, event-driven automation, and persistent session state across distributed development environments.

The platform distinguishes itself through its federated task management and policy-based access control, which
- [semantic-release/semantic-release](https://awesome-repositories.com/repository/semantic-release-semantic-release.md) (23,332 ⭐) — Semantic-release is an automated release management tool that determines version increments, generates changelogs, and publishes software packages by analyzing commit history against standardized conventions. It functions as a plugin-based orchestrator that integrates directly into continuous integration pipelines to manage the entire release lifecycle, from verifying environment conditions to distributing artifacts.

The project distinguishes itself through its commit-message-driven approach, which enforces consistent versioning standards and automates the creation of release notes based on t
- [drizzle-team/drizzle-orm](https://awesome-repositories.com/repository/drizzle-team-drizzle-orm.md) (34,835 ⭐) — Drizzle ORM is a TypeScript-native database toolkit providing type-safe SQL query building, schema management, and automated migrations across PostgreSQL, MySQL, SQLite, and SingleStore.
- [semantic-org/semantic-ui-react](https://awesome-repositories.com/repository/semantic-org-semantic-ui-react.md) (13,218 ⭐) — Semantic UI React is a declarative component library that provides native React bindings for the Semantic UI design language. It enables the construction of complex user interfaces through a modular, component-based architecture that maps directly to established design patterns, allowing developers to build consistent web application layouts without manual HTML markup.

The library distinguishes itself through a shorthand property system that automatically generates and populates nested child components from data objects, significantly reducing the need for verbose code. It also supports polym
- [appsilon/semantic.dashboard](https://awesome-repositories.com/repository/appsilon-semantic-dashboard.md) (256 ⭐) — semantic.dashboard
- [abetlen/llama-cpp-python](https://awesome-repositories.com/repository/abetlen-llama-cpp-python.md) (9,993 ⭐) — llama-cpp-python provides a Python interface for the llama.cpp library, enabling the execution of large language models with hardware acceleration. It functions as a GGUF model loader and a structured text generator capable of running inference servers and multimodal runtimes for processing both text and image inputs.

The project distinguishes itself through a local inference server that exposes model capabilities via an OpenAI-compatible web API. It supports advanced execution techniques including speculative decoding, weight quantization, and layer-based GPU offloading to manage memory acro
- [vibrantlabsai/ragas](https://awesome-repositories.com/repository/vibrantlabsai-ragas.md) (12,659 ⭐) — Ragas is an evaluation framework designed to measure the performance of retrieval-augmented generation pipelines and autonomous agent workflows. It provides a comprehensive suite of tools for benchmarking system outputs, utilizing language models as automated judges to score performance against defined rubrics and reference data. By standardizing inputs, retrieved contexts, and generated responses into a unified schema, the project enables consistent analysis across complex AI applications.

The framework distinguishes itself through its ability to generate synthetic test datasets from existin
- [semantic-org/semantic-ui](https://awesome-repositories.com/repository/semantic-org-semantic-ui.md) (51,064 ⭐) — Semantic-UI is an HTML and CSS UI framework consisting of a themed component library and a responsive layout framework. It provides a collection of reusable interface components and a grid-based system of columns and containers designed to build responsive websites.

The framework is distinguished by its use of natural-language class naming, which maps human-readable CSS classes to specific visual styles. It also functions as a right-to-left UI toolkit, utilizing directional mirroring to adjust visual flow and element alignment for languages read from right to left.

The system covers frontend
- [nvidia/semantic-segmentation](https://awesome-repositories.com/repository/nvidia-semantic-segmentation.md) (1,823 ⭐) — Nvidia Semantic Segmentation monorepo
- [rust-embedded-community/embedded-nal](https://awesome-repositories.com/repository/rust-embedded-community-embedded-nal.md) (202 ⭐) — An Embedded Network Abstraction Layer
- [vercel/ai](https://awesome-repositories.com/repository/vercel-ai.md) (21,885 ⭐) — This project is a comprehensive framework for building AI-powered applications, providing a unified toolkit for orchestrating language models, autonomous agents, and interactive user interfaces. It serves as a central library for managing the entire lifecycle of AI interactions, from initial prompt generation and model provider abstraction to complex, multi-step reasoning and tool execution.

The framework distinguishes itself through its deep integration with frontend development, specifically by enabling generative user interfaces that render dynamic components directly from model outputs. I
- [llmware-ai/llmware](https://awesome-repositories.com/repository/llmware-ai-llmware.md) (14,838 ⭐) — llmware is a Python framework for AI agent orchestration and model management, designed to coordinate multi-model workflows and autonomous agents. It provides a unified model catalog and standardized interface to execute specialized language models for complex research, analysis, and structured data generation.

The project distinguishes itself through its heavy emphasis on local execution and quantized inference, allowing models to run on private infrastructure using CPU, GPU, and NPU acceleration via runtimes like ONNX and OpenVino. It features a specialized ability to translate natural lang
- [cockroachdb/cockroach](https://awesome-repositories.com/repository/cockroachdb-cockroach.md) (32,207 ⭐) — Cockroach is a distributed SQL database designed to scale horizontally across multiple nodes while maintaining strict ACID compliance and global data consistency. It functions as a relational database engine that automatically partitions data into ranges, rebalancing them across a cluster to accommodate growing storage and throughput requirements. By utilizing a distributed consensus protocol, the system ensures that all nodes agree on the order of operations, providing fault tolerance and continuous availability even in the event of hardware failures.

The system distinguishes itself through
- [rust-embedded-community/embedded-storage](https://awesome-repositories.com/repository/rust-embedded-community-embedded-storage.md) (95 ⭐) — An Embedded Storage Abstraction Layer
- [aimerykong/recurrent-pixel-embedding-for-instance-grouping](https://awesome-repositories.com/repository/aimerykong-recurrent-pixel-embedding-for-instance-grouping.md) (146 ⭐) — CVPR2018 - pixel embedding & grouping for structured prediction, e.g., instance segmentation
- [aider-ai/aider](https://awesome-repositories.com/repository/aider-ai-aider.md) (46,305 ⭐) — Aider is a command-line interface tool that enables large language models to directly edit, refactor, and manage source code within a local repository. It functions as an AI-powered coding assistant that integrates into the developer workflow, allowing users to apply code changes through natural language prompts while maintaining repository context and version control.

The tool distinguishes itself through a specialized diff-based patching engine that parses model-generated search-and-replace blocks to modify specific file segments without rewriting entire files. It features a provider-agnost
- [zhayujie/chatgpt-on-wechat](https://awesome-repositories.com/repository/zhayujie-chatgpt-on-wechat.md) (45,353 ⭐) — This project is an autonomous agent framework designed to integrate large language models with popular messaging platforms. It functions as a middleware platform that enables automated, multimodal interactions by decomposing complex user goals into sequential plans, executing them through external tools, and maintaining persistent context across sessions.

The framework distinguishes itself through a modular skill architecture and a hybrid memory system. Users can extend system capabilities by installing custom logic modules from community hubs or generating them through natural language. The
- [anthropics/claude-code](https://awesome-repositories.com/repository/anthropics-claude-code.md) (132,728 ⭐) — Anthropic's terminal-native AI coding agent.
- [rust-embedded/awesome-embedded-rust](https://awesome-repositories.com/repository/rust-embedded-awesome-embedded-rust.md) (7,927 ⭐) — Curated list of resources for Embedded and Low-level development in the Rust programming language
- [zilliztech/gptcache](https://awesome-repositories.com/repository/zilliztech-gptcache.md) (8,068 ⭐) — GPTCache is a semantic caching layer and response optimizer for large language models. It functions as pluggable middleware for orchestration frameworks, utilizing vector database caching to store and retrieve model responses based on the semantic similarity of prompts rather than exact text matches.

The system uses embeddings to determine cache hits by comparing the distance between new queries and stored vectors. It employs a hybrid storage model that persists original prompts in relational databases while maintaining high-dimensional embeddings in vector stores.

The project covers a broad
- [lancedb/lancedb](https://awesome-repositories.com/repository/lancedb-lancedb.md) (9,031 ⭐) — LanceDB is a vector database and columnar data store designed to function as a versioned dataset manager and vector search engine. It serves as a high-performance backend for indexing and retrieving high-dimensional embeddings, providing the foundation for machine learning data pipelines.

The system distinguishes itself through a combination of cloud-native object storage and immutable version tracking, allowing for data time-travel and reproducible AI experiments. It integrates hybrid search capabilities, merging dense vector similarity with BM25 full-text search and SQL-like scalar filters
- [google-gemini/gemini-fullstack-langgraph-quickstart](https://awesome-repositories.com/repository/google-gemini-gemini-fullstack-langgraph-quickstart.md) (18,217 ⭐) — This project is an agentic workflow orchestrator designed for building and deploying autonomous systems that perform multi-step reasoning. It functions as a tool-augmented engine, enabling developers to chain model calls with external function execution to complete complex, user-defined tasks. By integrating large language models with persistent memory and stateful logic, the framework supports the creation of intelligent applications capable of independent operation.

The platform distinguishes itself through graph-based state orchestration, which allows developers to define logic steps and t
- [accelerated-text/accelerated-text](https://awesome-repositories.com/repository/accelerated-text-accelerated-text.md) (806 ⭐) — Accelerated Text is a no-code natural language generation platform. It will help you construct document plans which define how your data is converted to textual descriptions varying in wording and structure.
- [unstructured-io/unstructured](https://awesome-repositories.com/repository/unstructured-io-unstructured.md) (14,019 ⭐) — Unstructured is an enterprise-grade data orchestration engine designed to transform raw, unstructured files into structured, machine-readable formats. It functions as a comprehensive platform for document ingestion, partitioning, and enrichment, specifically engineered to prepare complex data for retrieval-augmented generation and agentic AI workflows.

The platform distinguishes itself through its sophisticated document processing strategies, which combine rule-based extraction with vision-language models to handle diverse file layouts, tables, and images. It provides a modular architecture t