Awesome GitHub RepositoriesPostgreSQL Vector Stores

This project is a privacy-first backend service designed to facilitate retrieval-augmented generation by processing local documents into searchable vector representations. It provides a modular architecture that allows users to ingest diverse file formats, manage document metadata, and perform semantic searches to provide context-aware responses for chat and completion requests. The system distinguishes itself through a database-agnostic abstraction layer that supports various storage backends, ranging from local disk storage to enterprise-grade vector databases. It offers flexible deployment

Vector is a high-performance observability data pipeline designed to collect, transform, and route logs, metrics, and traces across distributed infrastructure. It functions as a modular engine that decouples data ingestion from processing and transmission, utilizing a component-based architecture to connect diverse sources to multiple destinations. The project distinguishes itself through a focus on reliability and flow control. It implements backpressure-aware data movement to prevent data loss during traffic spikes and utilizes disk-backed event buffering to ensure durability during network

This project is a retrieval-augmented generation pipeline designed for building custom ChatGPT plugins that allow language models to query private or professional documents. It implements a full retrieval workflow, from processing and indexing document chunks to retrieving relevant context for natural language queries. The system distinguishes itself through a hybrid retrieval approach that combines dense vector embeddings with sparse keyword matching, further refined by a two-stage semantic re-ranking process. It includes specialized data privacy tools for screening personally identifiable i

Hub is a multimodal AI data lake and vector database designed for storing and querying embeddings, text, audio, and images. It functions as a dataset version control system and a machine learning data streaming engine to support large-scale model training. The system utilizes a serverless PostgreSQL vector store to index high-dimensional embeddings for semantic search. It provides a visual interface for inspecting multimodal datasets and viewing annotations such as bounding boxes and masks. The platform handles cloud-agnostic storage synchronization and implements lazy, compressed data strea

Arize Phoenix is an LLM observability platform and evaluation framework designed to capture execution traces and monitor large language model applications. It serves as a prompt management system for versioning and testing templates, and as a self-hosted AI operations infrastructure for managing telemetry and experiments. The platform differentiates itself through a specialized embedding visualization tool used to detect data drift and optimize vector search. It provides a comprehensive evaluation suite that utilizes judge-based evaluators and ground-truth datasets to score model outputs, and

pgloader is a command-line tool that automates the migration of data and schema from various source databases and file formats into PostgreSQL. It combines schema discovery, parallel data pipelines, and type casting into a single, declarative workflow, using PostgreSQL's COPY protocol for high-throughput bulk loading. The tool distinguishes itself by compiling a dedicated command language into concurrent reader-writer pipelines that handle schema introspection, data transformation, and error-resilient batch processing. It supports migrating entire databases from MySQL, MS SQL, SQLite, and Pos

GreptimeDB is a distributed, open-source time-series database built for unified observability. It stores and queries metrics, logs, and traces together in a single columnar engine, supporting both SQL and PromQL for analysis. The database is designed as a Kubernetes-native operator with a decoupled compute and storage architecture, enabling horizontal scaling and multi-region deployment. What distinguishes GreptimeDB is its role as a multi-protocol ingestion gateway, accepting data through OpenTelemetry, Prometheus Remote Write, InfluxDB, Loki, Elasticsearch, Kafka, and MQTT protocols without

Lamp Cloud is a multi-tenant SaaS backend framework built on Java and Spring Cloud that provides a complete foundation for building enterprise-grade administration systems. Its core identity centers on supporting multiple tenant isolation strategies—including database-per-tenant, schema-per-tenant, and shared-table modes—that can be switched without altering business code, alongside a role-based access control system enforced at the gateway layer across all microservices. The framework distinguishes itself through comprehensive tenant lifecycle management tools that allow creating, configurin

Weblate is an open-source web-based translation management system that provides a collaborative platform for teams to review, suggest, and approve translations in real time. It functions as a continuous localization platform, automatically synchronizing translations with source code changes in version control repositories, and can be deployed either as a self-hosted server or through a managed cloud hosting service. The system integrates directly with Git hosting platforms like GitHub, GitLab, and Bitbucket, storing all translations in version control with individual translator attribution re

Tortoise ORM is an asynchronous object-relational mapper for Python that mirrors Django's model and queryset API while running on asyncio. It defines database tables as Python classes with typed fields and supports foreign key, many-to-many, and one-to-one relations, providing a chainable query API for filtering, annotating, grouping, and prefetching related objects without blocking the event loop. The ORM includes a built-in migration engine that detects model changes, generates migration files, and applies or reverts schema changes through a command-line tool. It connects to PostgreSQL, MyS

wger is an open-source web application for fitness tracking, workout planning, and nutrition management. It provides a self-hosted platform where users can design weekly workout routines from a built-in exercise library, log their training progress, and plan daily meals using a food database with automatic nutritional calculations. The application supports multi-user accounts with credential-based login, passkey authentication, and third-party sign-in through OAuth providers. The platform includes a documented REST API that enables programmatic access to workout logs, meal plans, and user dat

CKAN is an open-source data management platform that provides the foundation for building data portals. It supports the full lifecycle of datasets—from creation and organization to publishing, cataloging with faceted search, and interactive data visualization—all through a web interface. The platform is built on a modular architecture that includes a plugin-based extensibility system, a harvesting framework for importing metadata from external sources, and a standardized RESTful JSON API for programmatic access to datasets and metadata. The web interface is rendered using the Jinja2 templatin

Django Silk is a profiling and inspection toolset for Django applications designed to capture SQL queries, HTTP request data, and execution timing for diagnostics. It functions as a performance profiler and debugging middleware that records runtime execution data to provide a comprehensive overview of application behavior. The system includes a database profiler for identifying slow operations through detailed timing data and an HTTP request inspector for reviewing headers, bodies, and network traffic via a web interface. It allows for the reproduction of specific server requests through gene

Athens es un servidor proxy de módulos de Go y caché de dependencias que proporciona un sistema de almacenamiento persistente para dependencias de Go. Actúa como un espejo y almacén de datos para garantizar entornos de construcción reproducibles almacenando copias inmutables de paquetes externos, protegiendo contra eliminaciones o interrupciones en el origen. El proyecto destaca por servir como una puerta de enlace segura para el alojamiento de módulos de Go privados, utilizando tokens de autenticación, claves SSH y GitHub Apps para recuperar dependencias de sistemas de control de versiones privados. Además, permite el cumplimiento de las dependencias de software mediante el filtrado de solicitudes y el proxy de sumas de comprobación, lo que evita que los metadatos de los módulos privados se filtren a servidores públicos. El servidor admite una amplia gama de backends de almacenamiento, incluyendo disco local, bases de datos NoSQL y almacenes de objetos en la nube compatibles con S3. Incluye capacidades para el almacenamiento en caché de dependencias distribuido con bloqueo compartido para evitar descargas redundantes a través de múltiples instancias y proporciona herramientas para el prellenado de almacenamiento en entornos aislados (air-gapped). El servidor puede desplegarse a través de contenedores Docker, gráficos Helm de Kubernetes o varias plataformas en la nube gestionadas.

ruvector es un almacén de vectores y base de datos de grafos basado en Rust, diseñado para inferencia local y búsquedas de vecinos más cercanos. Utiliza una arquitectura de base de datos de grafos vectoriales y un índice de red neuronal de grafos para refinar los rankings de búsqueda mediante atención estructural. El sistema incluye un simulador de circuitos cuánticos acelerado por hardware para ejecutar simulaciones de vectores de estado y patrones de búsqueda complejos, junto con un motor de inferencia WebAssembly para ejecutar búsquedas vectoriales y modelos directamente en navegadores web. El proyecto emplea un formato de contenedor cognitivo que agrupa modelos, datos y un microkernel arrancable en un único binario para su despliegue. Incluye herramientas especializadas de configuración de modelos, como un método de consolidación de pesos para prevenir el olvido catastrófico y un mecanismo de adaptadores ligeros para la adaptación instantánea de pesos. El sistema cubre una amplia superficie de capacidades, incluyendo búsqueda vectorial acelerada por hardware, consultas de relaciones en grafos y análisis de documentos científicos para la extracción de LaTeX y MathML. También proporciona encadenamiento de testigos criptográficos para verificar mutaciones de datos, sincronización de metadatos basada en Raft para alta disponibilidad y compresión de datos de resolución escalonada para gestionar los costes de almacenamiento.

Chonkie es una librería de fragmentación de texto (chunking) diseñada para pipelines de generación aumentada por recuperación (RAG). Funciona como un divisor de texto semántico y pipeline de ingesta RAG, transformando texto sin procesar en segmentos incrustados para su almacenamiento en bases de datos vectoriales. El proyecto se distingue por estrategias de división especializadas, incluyendo un divisor de código basado en AST para preservar límites lógicos en el código fuente y un divisor de texto semántico que utiliza modelos de embedding para determinar límites basados en el significado. También proporciona un ingestor de bases de datos vectoriales para automatizar la generación de embeddings y su exportación a varios almacenes. La librería cubre una amplia gama de capacidades, incluyendo el análisis de documentos mediante OCR y extracción de markdown, una variedad de métodos de división como conteo de tokens y segmentación jerárquica, y orquestación de flujos de trabajo a través de pipelines reutilizables. Admite una amplia gama de integraciones de almacenes vectoriales, incluyendo Qdrant, Milvus, Weaviate y Elasticsearch, así como la exportación de datos a JSON y datasets de Hugging Face. Los usuarios pueden ejecutar estas operaciones a través de una interfaz de línea de comandos o desplegar el sistema como un servicio API contenerizado.

Bazarr is an automated subtitle management system and downloader designed to discover, acquire, and synchronize subtitles for movies and TV shows. It functions as a media library companion that integrates with external media managers and servers via APIs to track missing subtitles and ensure libraries are up to date. The project distinguishes itself through advanced media processing, using neural-network audio transcription to generate subtitles from audio tracks or translate foreign dialogue into English. It also features audio-based synchronization to align subtitle timing with video conten

Langroid is a multi-agent orchestration framework and tool integration suite designed for building complex AI applications. It serves as a multi-modal integration layer that connects diverse local and remote language models with an agentic retrieval-augmented generation system. The project distinguishes itself through a collaborative message-exchange paradigm, allowing specialized agents to delegate tasks hierarchically and coordinate via structured communication. It features an advanced state management system for conversational AI, including the ability to rewind and prune conversation hist

Gravitino is a federated metadata lake and unified data catalog designed to manage tables, files, and AI models across diverse data sources and cloud storage. It serves as a centralized interface for governing schemas, access controls, and tagging across relational databases, messaging queues, and object stores. The project distinguishes itself by unifying the management of AI assets, such as machine learning models and their version lineages, alongside traditional tabular data. It also implements the Iceberg REST specification to provide a standardized metadata server and proxy for lakehouse