Awesome GitHub RepositoriesPostgreSQL Vector Stores

This project is a privacy-first backend service designed to facilitate retrieval-augmented generation by processing local documents into searchable vector representations. It provides a modular architecture that allows users to ingest diverse file formats, manage document metadata, and perform semantic searches to provide context-aware responses for chat and completion requests. The system distinguishes itself through a database-agnostic abstraction layer that supports various storage backends, ranging from local disk storage to enterprise-grade vector databases. It offers flexible deployment

Vector is a high-performance observability data pipeline designed to collect, transform, and route logs, metrics, and traces across distributed infrastructure. It functions as a modular engine that decouples data ingestion from processing and transmission, utilizing a component-based architecture to connect diverse sources to multiple destinations. The project distinguishes itself through a focus on reliability and flow control. It implements backpressure-aware data movement to prevent data loss during traffic spikes and utilizes disk-backed event buffering to ensure durability during network

This project is a retrieval-augmented generation pipeline designed for building custom ChatGPT plugins that allow language models to query private or professional documents. It implements a full retrieval workflow, from processing and indexing document chunks to retrieving relevant context for natural language queries. The system distinguishes itself through a hybrid retrieval approach that combines dense vector embeddings with sparse keyword matching, further refined by a two-stage semantic re-ranking process. It includes specialized data privacy tools for screening personally identifiable i

Hub is a multimodal AI data lake and vector database designed for storing and querying embeddings, text, audio, and images. It functions as a dataset version control system and a machine learning data streaming engine to support large-scale model training. The system utilizes a serverless PostgreSQL vector store to index high-dimensional embeddings for semantic search. It provides a visual interface for inspecting multimodal datasets and viewing annotations such as bounding boxes and masks. The platform handles cloud-agnostic storage synchronization and implements lazy, compressed data strea

Arize Phoenix is an LLM observability platform and evaluation framework designed to capture execution traces and monitor large language model applications. It serves as a prompt management system for versioning and testing templates, and as a self-hosted AI operations infrastructure for managing telemetry and experiments. The platform differentiates itself through a specialized embedding visualization tool used to detect data drift and optimize vector search. It provides a comprehensive evaluation suite that utilizes judge-based evaluators and ground-truth datasets to score model outputs, and

pgloader is a command-line tool that automates the migration of data and schema from various source databases and file formats into PostgreSQL. It combines schema discovery, parallel data pipelines, and type casting into a single, declarative workflow, using PostgreSQL's COPY protocol for high-throughput bulk loading. The tool distinguishes itself by compiling a dedicated command language into concurrent reader-writer pipelines that handle schema introspection, data transformation, and error-resilient batch processing. It supports migrating entire databases from MySQL, MS SQL, SQLite, and Pos

GreptimeDB is a distributed, open-source time-series database built for unified observability. It stores and queries metrics, logs, and traces together in a single columnar engine, supporting both SQL and PromQL for analysis. The database is designed as a Kubernetes-native operator with a decoupled compute and storage architecture, enabling horizontal scaling and multi-region deployment. What distinguishes GreptimeDB is its role as a multi-protocol ingestion gateway, accepting data through OpenTelemetry, Prometheus Remote Write, InfluxDB, Loki, Elasticsearch, Kafka, and MQTT protocols without

Lamp Cloud is a multi-tenant SaaS backend framework built on Java and Spring Cloud that provides a complete foundation for building enterprise-grade administration systems. Its core identity centers on supporting multiple tenant isolation strategies—including database-per-tenant, schema-per-tenant, and shared-table modes—that can be switched without altering business code, alongside a role-based access control system enforced at the gateway layer across all microservices. The framework distinguishes itself through comprehensive tenant lifecycle management tools that allow creating, configurin

Weblate is an open-source web-based translation management system that provides a collaborative platform for teams to review, suggest, and approve translations in real time. It functions as a continuous localization platform, automatically synchronizing translations with source code changes in version control repositories, and can be deployed either as a self-hosted server or through a managed cloud hosting service. The system integrates directly with Git hosting platforms like GitHub, GitLab, and Bitbucket, storing all translations in version control with individual translator attribution re

Tortoise ORM is an asynchronous object-relational mapper for Python that mirrors Django's model and queryset API while running on asyncio. It defines database tables as Python classes with typed fields and supports foreign key, many-to-many, and one-to-one relations, providing a chainable query API for filtering, annotating, grouping, and prefetching related objects without blocking the event loop. The ORM includes a built-in migration engine that detects model changes, generates migration files, and applies or reverts schema changes through a command-line tool. It connects to PostgreSQL, MyS

wger is an open-source web application for fitness tracking, workout planning, and nutrition management. It provides a self-hosted platform where users can design weekly workout routines from a built-in exercise library, log their training progress, and plan daily meals using a food database with automatic nutritional calculations. The application supports multi-user accounts with credential-based login, passkey authentication, and third-party sign-in through OAuth providers. The platform includes a documented REST API that enables programmatic access to workout logs, meal plans, and user dat

CKAN is an open-source data management platform that provides the foundation for building data portals. It supports the full lifecycle of datasets—from creation and organization to publishing, cataloging with faceted search, and interactive data visualization—all through a web interface. The platform is built on a modular architecture that includes a plugin-based extensibility system, a harvesting framework for importing metadata from external sources, and a standardized RESTful JSON API for programmatic access to datasets and metadata. The web interface is rendered using the Jinja2 templatin

Django Silk is a profiling and inspection toolset for Django applications designed to capture SQL queries, HTTP request data, and execution timing for diagnostics. It functions as a performance profiler and debugging middleware that records runtime execution data to provide a comprehensive overview of application behavior. The system includes a database profiler for identifying slow operations through detailed timing data and an HTTP request inspector for reviewing headers, bodies, and network traffic via a web interface. It allows for the reproduction of specific server requests through gene

Athens is a Go module proxy server and dependency cache that provides a persistent storage system for Go dependencies. It acts as a mirror and datastore to ensure reproducible build environments by storing immutable copies of external packages, protecting against upstream deletions or outages. The project distinguishes itself by serving as a secure gateway for private Go module hosting, utilizing authentication tokens, SSH keys, and GitHub Apps to retrieve dependencies from private version control systems. It further enables software dependency compliance through request filtering and checksu

ruvector 是一个基于 Rust 的向量存储和图数据库，专为本地推理和最近邻搜索而设计。它利用向量图数据库架构和图神经网络索引，通过结构化注意力机制优化搜索排名。该系统包括用于执行状态向量模拟和复杂搜索模式的硬件加速量子电路模拟器，以及用于直接在 Web 浏览器中运行向量搜索和模型执行的 WebAssembly 推理引擎。 该项目采用认知容器格式，将模型、数据和可启动微内核打包成单个二进制文件进行部署。它具有专门的模型配置工具，包括防止灾难性遗忘的权重合并方法，以及用于即时权重适配的轻量级适配器机制。 该系统涵盖了广泛的功能面，包括硬件加速向量搜索、图关系查询以及用于 LaTeX 和 MathML 提取的科学文档解析。它还提供用于验证数据变更的加密见证链、基于 Raft 的高可用元数据同步，以及用于管理存储成本的分层分辨率数据压缩。

Chonkie 是一个专为检索增强生成 (RAG) 流水线设计的文本分块库。它充当语义文本分割器和 RAG 数据摄取流水线，将原始文本转换为嵌入片段，以便存储在向量数据库中。 该项目通过专门的分割策略脱颖而出，包括用于保留源代码逻辑边界的基于 AST 的代码分割器，以及使用嵌入模型根据语义确定边界的语义文本分割器。它还提供了一个向量数据库摄取器，用于自动化生成嵌入并将其导出到各种存储中。 该库涵盖了广泛的功能，包括通过 OCR 和 Markdown 提取进行文档解析，多种分割方法（如基于 Token 计数和分层分割），以及通过可重用流水线进行工作流编排。它支持多种向量存储集成，包括 Qdrant、Milvus、Weaviate 和 Elasticsearch，以及将数据导出为 JSON 和 Hugging Face 数据集。 用户可以通过命令行界面执行这些操作，或将系统部署为容器化的 API 服务。

Bazarr 是一个自动化字幕管理系统和下载器，旨在发现、获取和同步电影和电视节目的字幕。它作为一个媒体库伴侣，通过 API 与外部媒体管理器和服务器集成，以跟踪缺失的字幕并确保库保持最新。 该项目通过高级媒体处理脱颖而出，使用神经网络音频转录从音轨生成字幕或将外语对话翻译成英语。它还具有基于音频的同步功能，可将字幕时间与视频内容对齐，并能在发现更高质量版本时自动升级现有字幕。 该系统涵盖了广泛的自动化功能，包括语言配置文件管理、跨多个字幕提供商的自适应搜索调度以及事件驱动的 Webhook 触发器。它支持通过 SQLite 或 PostgreSQL 进行数据持久化，并包括用于定位跨不同主机系统或容器的媒体文件的路径映射翻译工具。 该应用可以作为 Windows 上的后台服务或通过其他平台上的系统初始化管理器部署，并支持用于远程访问的反向代理配置。

Langroid is a multi-agent orchestration framework and tool integration suite designed for building complex AI applications. It serves as a multi-modal integration layer that connects diverse local and remote language models with an agentic retrieval-augmented generation system. The project distinguishes itself through a collaborative message-exchange paradigm, allowing specialized agents to delegate tasks hierarchically and coordinate via structured communication. It features an advanced state management system for conversational AI, including the ability to rewind and prune conversation hist

Gravitino is a federated metadata lake and unified data catalog designed to manage tables, files, and AI models across diverse data sources and cloud storage. It serves as a centralized interface for governing schemas, access controls, and tagging across relational databases, messaging queues, and object stores. The project distinguishes itself by unifying the management of AI assets, such as machine learning models and their version lineages, alongside traditional tabular data. It also implements the Iceberg REST specification to provide a standardized metadata server and proxy for lakehouse