Awesome GitHub RepositoriesEmbedding Models

This repository serves as a comprehensive library of architectural blueprints and code examples for integrating large language models into software applications. It functions as a developer learning resource, providing structured tutorials and implementation patterns that demonstrate how to build intelligent features using advanced prompting and data processing techniques. The collection distinguishes itself by focusing on complex reasoning and data-grounding workflows. It provides practical guidance on implementing retrieval-augmented generation pipelines, which connect language models to pr

This project is an autonomous agent framework designed to integrate large language models with popular messaging platforms. It functions as a middleware platform that enables automated, multimodal interactions by decomposing complex user goals into sequential plans, executing them through external tools, and maintaining persistent context across sessions. The framework distinguishes itself through a modular skill architecture and a hybrid memory system. Users can extend system capabilities by installing custom logic modules from community hubs or generating them through natural language. The

This project is a feature-rich Go client library designed for interacting with Redis. It serves as a comprehensive interface for managing remote data stores, enabling developers to execute standard database commands, handle complex data structures, and perform asynchronous operations within Go applications. The library distinguishes itself through its support for advanced Redis capabilities, including connection pooling, pipelining, and transactional integrity. It provides specialized primitives for managing distributed clusters, including automated topology updates and request routing to sha

This project is an agentic workflow orchestrator designed for building and deploying autonomous systems that perform multi-step reasoning. It functions as a tool-augmented engine, enabling developers to chain model calls with external function execution to complete complex, user-defined tasks. By integrating large language models with persistent memory and stateful logic, the framework supports the creation of intelligent applications capable of independent operation. The platform distinguishes itself through graph-based state orchestration, which allows developers to define logic steps and t

LangBot is an orchestration platform designed for building, managing, and deploying AI agents. It functions as a comprehensive framework for integrating large language models with custom workflows, enabling developers to connect intelligent agents to various messaging platforms and external tools. The platform distinguishes itself through a modular, plugin-based architecture that allows for the extension of agent capabilities via custom tools and file parsers. It features a secure, sandbox-isolated runtime environment that executes untrusted code and plugin logic within resource-constrained c

LanceDB is a vector database and columnar data store designed to function as a versioned dataset manager and vector search engine. It serves as a high-performance backend for indexing and retrieving high-dimensional embeddings, providing the foundation for machine learning data pipelines. The system distinguishes itself through a combination of cloud-native object storage and immutable version tracking, allowing for data time-travel and reproducible AI experiments. It integrates hybrid search capabilities, merging dense vector similarity with BM25 full-text search and SQL-like scalar filters

BERTopic is a topic modeling library used to extract interpretable themes from collections of text documents and images. It functions as a document clustering framework that transforms unstructured data into numerical vectors to group semantically similar content. The project distinguishes itself through a multimodal embedding tool that allows for joint clustering of text and images in a shared vector space. It also features a class-based TF-IDF representation engine to identify representative words for clusters and an integrated system for using large language models to generate natural lang

Torchtune is a PyTorch-native library for fine-tuning, aligning, and quantizing large language models. It provides a config-driven system for instantiating components, orchestrating distributed training, and managing parameter-efficient fine-tuning with quantization support, all through YAML-based configurations and command-line overrides. The library distinguishes itself through its comprehensive post-training workflow orchestration, combining supervised fine-tuning, preference optimization (DPO, PPO, GRPO), knowledge distillation, and quantization-aware training in a single configurable pip

Torchtune is a PyTorch-native library for fine-tuning, aligning, and quantizing large language models. It provides a configurable training pipeline orchestrated through YAML recipes, with CLI overrides and component swapping, distributed training via FSDP2, memory optimizations, and parameter-efficient fine-tuning methods like LoRA, DoRA, and QLoRA. The library distinguishes itself through its YAML-driven configuration system that defines all training parameters and instantiates components from config files, with full CLI override capability for any field or component at launch time. It suppo

LLMLingua is a prompt compression tool that reduces token count in prompts before they are sent to a large language model, cutting API costs and latency while preserving task performance. It operates as an extractive pipeline using a BERT-level Transformer encoder to classify each token for removal based on full bidirectional context from the prompt, retaining only key information and discarding non-essential tokens. The tool is trained through a knowledge distillation process, where a compact compression model learns from an extractive dataset derived from a large language model's output to

text2vec 是一个文本向量化工具包和语义相似度框架，用于将单词和句子转换为数值向量。它提供了用于生成嵌入、计算语义接近度以及实现词法和语义搜索的集成工具集。 该项目包括一个模型微调流水线，用于使用监督或无监督数据集优化嵌入和匹配模型。它还通过提供文本嵌入 API 进一步脱颖而出，该 API 允许通过 gRPC 或 HTTP 协议将向量化模型部署为网络服务。 该框架涵盖了广泛的功能，包括语义分析、神经文本编码和向量搜索检索。它支持密集向量嵌入和诸如 BM25 之类的词项重叠算法，以确定相似度并识别相关词项。 额外的性能优化包括模型蒸馏、基于 PCA 的降维以及针对大规模数据集的多 GPU 分布式推理。

fast-reid 是一个基于 PyTorch 的计算机视觉框架，旨在构建、训练和部署用于基于身份的视觉任务的深度学习模型。它提供了一个用于行人重识别和车辆重识别的专用工具箱，能够跨非重叠的摄像机视图匹配个人和车辆。 该项目包括用于识别特定身体特征和属性的行人属性识别工具。它具有一个模块化模型库，允许交换和基准测试不同的重识别架构。 该框架涵盖了大规模开发基础设施，包括跨多个 GPU 的分布式训练、混合精度训练，以及将表示从复杂网络转移到较小学生模型的知识蒸馏。它还提供了一个超参数优化循环、多数据集评估引擎，以及用于将模型导出为行业标准格式以进行生产部署的管道。

nano-graphrag 是一个检索系统，使用知识图谱为大语言模型响应提供结构化上下文。它既是一个将非结构化文本转换为实体和关系网络的知识图谱索引器，也是一个混合图检索系统。 该项目通过结合局部邻域搜索和全局社区摘要来回答复杂的自然语言问题，从而脱颖而出。它包含一个知识图谱可视化工具，可生成实体及其关系的 HTML 表示，以映射索引知识。 该框架涵盖了广泛的功能，包括实体关系提取、基于社区的图聚类和基于哈希的增量索引。它提供了一个集成层，用于连接开源模型和本地嵌入提供程序，并支持用于键值、向量和图数据的可插拔存储后端。通过基于参数的响应缓存和用于修复语言模型不稳定 JSON 输出的后处理函数，提供了额外的实用性。

FastVideo is a comprehensive system for accelerated video generation, serving as a video generation inference engine, a video diffusion training framework, and a modular pipeline orchestrator. It provides a distributed transformer optimizer and a distillation toolkit designed to reduce denoising steps and model complexity to increase frame rates. The project distinguishes itself through specialized acceleration techniques, including joint distillation and sparse attention training. It implements low-step video generation and weight quantization to FP8 or FP4 precision to increase throughput a

MIRIX is an AI agent state orchestrator and long-term memory system designed to provide persistent context for large language models. It functions as a multi-modal AI memory pipeline that processes text, voice, and screen captures into structured knowledge stores, including a dedicated screen activity knowledge base. The project distinguishes itself by integrating a multi-modal observation pipeline that monitors desktop activity in real-time to build a searchable history of user actions. It utilizes a multi-tiered memory hierarchy—separating episodic, semantic, procedural, and core stores—and

Knowledge-Distillation-Zoo is a framework for neural network model compression that facilitates the transfer of learned patterns from large teacher models to smaller student architectures. It provides a modular environment for executing training pipelines designed to reduce the computational requirements of deep learning models while maintaining predictive accuracy. The library implements knowledge transfer through both logit-based mimicry and feature-map alignment, allowing students to replicate the classification behavior and internal representations of a teacher. It supports teacher-studen

Superlinked is a development framework designed for building semantic search and retrieval pipelines. It functions as a machine learning data pipeline and semantic retrieval engine, providing the tools necessary to unify data schema definition, embedding generation, and vector database integration within a single application. The framework distinguishes itself by acting as a vector database orchestrator that manages the lifecycle of machine learning models alongside complex search logic. It enables developers to construct structured data models that map raw content and metadata into unified r