Awesome GitHub RepositoriesPostgreSQL Vector Stores

This project is a privacy-first backend service designed to facilitate retrieval-augmented generation by processing local documents into searchable vector representations. It provides a modular architecture that allows users to ingest diverse file formats, manage document metadata, and perform semantic searches to provide context-aware responses for chat and completion requests. The system distinguishes itself through a database-agnostic abstraction layer that supports various storage backends, ranging from local disk storage to enterprise-grade vector databases. It offers flexible deployment

Vector is a high-performance observability data pipeline designed to collect, transform, and route logs, metrics, and traces across distributed infrastructure. It functions as a modular engine that decouples data ingestion from processing and transmission, utilizing a component-based architecture to connect diverse sources to multiple destinations. The project distinguishes itself through a focus on reliability and flow control. It implements backpressure-aware data movement to prevent data loss during traffic spikes and utilizes disk-backed event buffering to ensure durability during network

This project is a retrieval-augmented generation pipeline designed for building custom ChatGPT plugins that allow language models to query private or professional documents. It implements a full retrieval workflow, from processing and indexing document chunks to retrieving relevant context for natural language queries. The system distinguishes itself through a hybrid retrieval approach that combines dense vector embeddings with sparse keyword matching, further refined by a two-stage semantic re-ranking process. It includes specialized data privacy tools for screening personally identifiable i

Hub is a multimodal AI data lake and vector database designed for storing and querying embeddings, text, audio, and images. It functions as a dataset version control system and a machine learning data streaming engine to support large-scale model training. The system utilizes a serverless PostgreSQL vector store to index high-dimensional embeddings for semantic search. It provides a visual interface for inspecting multimodal datasets and viewing annotations such as bounding boxes and masks. The platform handles cloud-agnostic storage synchronization and implements lazy, compressed data strea

Arize Phoenix is an LLM observability platform and evaluation framework designed to capture execution traces and monitor large language model applications. It serves as a prompt management system for versioning and testing templates, and as a self-hosted AI operations infrastructure for managing telemetry and experiments. The platform differentiates itself through a specialized embedding visualization tool used to detect data drift and optimize vector search. It provides a comprehensive evaluation suite that utilizes judge-based evaluators and ground-truth datasets to score model outputs, and

pgloader is a command-line tool that automates the migration of data and schema from various source databases and file formats into PostgreSQL. It combines schema discovery, parallel data pipelines, and type casting into a single, declarative workflow, using PostgreSQL's COPY protocol for high-throughput bulk loading. The tool distinguishes itself by compiling a dedicated command language into concurrent reader-writer pipelines that handle schema introspection, data transformation, and error-resilient batch processing. It supports migrating entire databases from MySQL, MS SQL, SQLite, and Pos

GreptimeDB is a distributed, open-source time-series database built for unified observability. It stores and queries metrics, logs, and traces together in a single columnar engine, supporting both SQL and PromQL for analysis. The database is designed as a Kubernetes-native operator with a decoupled compute and storage architecture, enabling horizontal scaling and multi-region deployment. What distinguishes GreptimeDB is its role as a multi-protocol ingestion gateway, accepting data through OpenTelemetry, Prometheus Remote Write, InfluxDB, Loki, Elasticsearch, Kafka, and MQTT protocols without

Lamp Cloud is a multi-tenant SaaS backend framework built on Java and Spring Cloud that provides a complete foundation for building enterprise-grade administration systems. Its core identity centers on supporting multiple tenant isolation strategies—including database-per-tenant, schema-per-tenant, and shared-table modes—that can be switched without altering business code, alongside a role-based access control system enforced at the gateway layer across all microservices. The framework distinguishes itself through comprehensive tenant lifecycle management tools that allow creating, configurin

Weblate is an open-source web-based translation management system that provides a collaborative platform for teams to review, suggest, and approve translations in real time. It functions as a continuous localization platform, automatically synchronizing translations with source code changes in version control repositories, and can be deployed either as a self-hosted server or through a managed cloud hosting service. The system integrates directly with Git hosting platforms like GitHub, GitLab, and Bitbucket, storing all translations in version control with individual translator attribution re

Tortoise ORM is an asynchronous object-relational mapper for Python that mirrors Django's model and queryset API while running on asyncio. It defines database tables as Python classes with typed fields and supports foreign key, many-to-many, and one-to-one relations, providing a chainable query API for filtering, annotating, grouping, and prefetching related objects without blocking the event loop. The ORM includes a built-in migration engine that detects model changes, generates migration files, and applies or reverts schema changes through a command-line tool. It connects to PostgreSQL, MyS

wger is an open-source web application for fitness tracking, workout planning, and nutrition management. It provides a self-hosted platform where users can design weekly workout routines from a built-in exercise library, log their training progress, and plan daily meals using a food database with automatic nutritional calculations. The application supports multi-user accounts with credential-based login, passkey authentication, and third-party sign-in through OAuth providers. The platform includes a documented REST API that enables programmatic access to workout logs, meal plans, and user dat

CKAN is an open-source data management platform that provides the foundation for building data portals. It supports the full lifecycle of datasets—from creation and organization to publishing, cataloging with faceted search, and interactive data visualization—all through a web interface. The platform is built on a modular architecture that includes a plugin-based extensibility system, a harvesting framework for importing metadata from external sources, and a standardized RESTful JSON API for programmatic access to datasets and metadata. The web interface is rendered using the Jinja2 templatin

Django Silk is a profiling and inspection toolset for Django applications designed to capture SQL queries, HTTP request data, and execution timing for diagnostics. It functions as a performance profiler and debugging middleware that records runtime execution data to provide a comprehensive overview of application behavior. The system includes a database profiler for identifying slow operations through detailed timing data and an HTTP request inspector for reviewing headers, bodies, and network traffic via a web interface. It allows for the reproduction of specific server requests through gene

Athens is a Go module proxy server and dependency cache that provides a persistent storage system for Go dependencies. It acts as a mirror and datastore to ensure reproducible build environments by storing immutable copies of external packages, protecting against upstream deletions or outages. The project distinguishes itself by serving as a secure gateway for private Go module hosting, utilizing authentication tokens, SSH keys, and GitHub Apps to retrieve dependencies from private version control systems. It further enables software dependency compliance through request filtering and checksu

ruvector este un vector store și o bază de date graf bazată pe Rust, concepută pentru inferență locală și căutări de tip nearest neighbor. Utilizează o arhitectură de bază de date graf vectorială și un index de rețele neuronale grafice pentru a rafina clasamentele de căutare prin atenție structurală. Sistemul include un simulator de circuite cuantice accelerat hardware pentru execuția simulărilor de tip state-vector și a modelelor complexe de căutare, alături de un motor de inferență WebAssembly pentru rularea căutărilor vectoriale și execuția modelelor direct în browserele web. Proiectul folosește un format de container cognitiv care grupează modelele, datele și un microkernel bootabil într-un singur binar pentru deployment. Acesta dispune de instrumente specializate de configurare a modelelor, inclusiv o metodă de consolidare a ponderilor pentru a preveni uitarea catastrofală și un mecanism de adaptare ușor pentru ajustarea instantanee a ponderilor. Sistemul acoperă o gamă largă de capabilități, inclusiv căutarea vectorială accelerată hardware, interogarea relațiilor grafice și parsarea documentelor științifice pentru extragerea LaTeX și MathML. De asemenea, oferă înlănțuire de dovezi criptografice pentru verificarea modificărilor de date, sincronizarea metadatelor bazată pe Raft pentru disponibilitate ridicată și compresia datelor cu rezoluție pe niveluri pentru gestionarea costurilor de stocare.

Chonkie este o bibliotecă de chunking (segmentare) a textului concepută pentru pipeline-uri de retrieval-augmented generation (RAG). Funcționează ca un splitter semantic de text și un pipeline de ingestie RAG, transformând textul brut în segmente încorporate pentru stocare în baze de date vectoriale. Proiectul se distinge prin strategii de segmentare specializate, inclusiv un splitter de cod bazat pe AST pentru păstrarea limitelor logice în codul sursă și un splitter semantic de text care utilizează modele de embedding pentru a determina limitele bazate pe semnificație. De asemenea, oferă un ingestor pentru baze de date vectoriale pentru a automatiza generarea embedding-urilor și exportul acestora către diverse stocuri. Biblioteca acoperă o gamă largă de capabilități, inclusiv parsarea documentelor prin OCR și extragerea markdown, o varietate de metode de segmentare precum numărarea token-urilor și segmentarea ierarhică, și orchestrarea fluxului de lucru prin pipeline-uri reutilizabile. Suportă o gamă largă de integrări cu vector store-uri, inclusiv Qdrant, Milvus, Weaviate și Elasticsearch, precum și exportul datelor către JSON și seturi de date Hugging Face. Utilizatorii pot executa aceste operațiuni printr-o interfață în linie de comandă sau pot implementa sistemul ca serviciu API containerizat.

Bazarr este un sistem automat de gestionare și descărcare a subtitrărilor, conceput pentru a descoperi, achiziționa și sincroniza subtitrări pentru filme și emisiuni TV. Funcționează ca un companion pentru bibliotecile media care se integrează cu manageri și servere media externe prin API-uri pentru a urmări subtitrările lipsă și a asigura actualizarea bibliotecilor. Proiectul se distinge prin procesarea avansată a conținutului media, folosind transcrierea audio prin rețele neuronale pentru a genera subtitrări din piste audio sau pentru a traduce dialogurile străine în engleză. De asemenea, dispune de sincronizare bazată pe audio pentru a alinia sincronizarea subtitrărilor cu conținutul video și poate actualiza automat subtitrările existente atunci când sunt descoperite versiuni de calitate superioară. Sistemul acoperă o gamă largă de capabilități de automatizare, inclusiv gestionarea profilurilor de limbă, programarea căutărilor adaptive prin mai mulți furnizori de subtitrări și declanșatoare de tip webhook bazate pe evenimente. Suportă persistența datelor prin SQLite sau PostgreSQL și include instrumente pentru traducerea mapării căilor pentru a localiza fișierele media pe diferite sisteme gazdă sau containere. Aplicația poate fi implementată ca serviciu de fundal pe Windows sau prin manageri de inițializare a sistemului pe alte platforme, cu suport pentru configurarea proxy-ului invers pentru acces la distanță.

Langroid is a multi-agent orchestration framework and tool integration suite designed for building complex AI applications. It serves as a multi-modal integration layer that connects diverse local and remote language models with an agentic retrieval-augmented generation system. The project distinguishes itself through a collaborative message-exchange paradigm, allowing specialized agents to delegate tasks hierarchically and coordinate via structured communication. It features an advanced state management system for conversational AI, including the ability to rewind and prune conversation hist

Gravitino is a federated metadata lake and unified data catalog designed to manage tables, files, and AI models across diverse data sources and cloud storage. It serves as a centralized interface for governing schemas, access controls, and tagging across relational databases, messaging queues, and object stores. The project distinguishes itself by unifying the management of AI assets, such as machine learning models and their version lineages, alongside traditional tabular data. It also implements the Iceberg REST specification to provide a standardized metadata server and proxy for lakehouse