Awesome GitHub RepositoriesPostgreSQL Vector Stores

This project is a privacy-first backend service designed to facilitate retrieval-augmented generation by processing local documents into searchable vector representations. It provides a modular architecture that allows users to ingest diverse file formats, manage document metadata, and perform semantic searches to provide context-aware responses for chat and completion requests. The system distinguishes itself through a database-agnostic abstraction layer that supports various storage backends, ranging from local disk storage to enterprise-grade vector databases. It offers flexible deployment

Vector is a high-performance observability data pipeline designed to collect, transform, and route logs, metrics, and traces across distributed infrastructure. It functions as a modular engine that decouples data ingestion from processing and transmission, utilizing a component-based architecture to connect diverse sources to multiple destinations. The project distinguishes itself through a focus on reliability and flow control. It implements backpressure-aware data movement to prevent data loss during traffic spikes and utilizes disk-backed event buffering to ensure durability during network

This project is a retrieval-augmented generation pipeline designed for building custom ChatGPT plugins that allow language models to query private or professional documents. It implements a full retrieval workflow, from processing and indexing document chunks to retrieving relevant context for natural language queries. The system distinguishes itself through a hybrid retrieval approach that combines dense vector embeddings with sparse keyword matching, further refined by a two-stage semantic re-ranking process. It includes specialized data privacy tools for screening personally identifiable i

Hub is a multimodal AI data lake and vector database designed for storing and querying embeddings, text, audio, and images. It functions as a dataset version control system and a machine learning data streaming engine to support large-scale model training. The system utilizes a serverless PostgreSQL vector store to index high-dimensional embeddings for semantic search. It provides a visual interface for inspecting multimodal datasets and viewing annotations such as bounding boxes and masks. The platform handles cloud-agnostic storage synchronization and implements lazy, compressed data strea

Arize Phoenix is an LLM observability platform and evaluation framework designed to capture execution traces and monitor large language model applications. It serves as a prompt management system for versioning and testing templates, and as a self-hosted AI operations infrastructure for managing telemetry and experiments. The platform differentiates itself through a specialized embedding visualization tool used to detect data drift and optimize vector search. It provides a comprehensive evaluation suite that utilizes judge-based evaluators and ground-truth datasets to score model outputs, and

pgloader is a command-line tool that automates the migration of data and schema from various source databases and file formats into PostgreSQL. It combines schema discovery, parallel data pipelines, and type casting into a single, declarative workflow, using PostgreSQL's COPY protocol for high-throughput bulk loading. The tool distinguishes itself by compiling a dedicated command language into concurrent reader-writer pipelines that handle schema introspection, data transformation, and error-resilient batch processing. It supports migrating entire databases from MySQL, MS SQL, SQLite, and Pos

GreptimeDB is a distributed, open-source time-series database built for unified observability. It stores and queries metrics, logs, and traces together in a single columnar engine, supporting both SQL and PromQL for analysis. The database is designed as a Kubernetes-native operator with a decoupled compute and storage architecture, enabling horizontal scaling and multi-region deployment. What distinguishes GreptimeDB is its role as a multi-protocol ingestion gateway, accepting data through OpenTelemetry, Prometheus Remote Write, InfluxDB, Loki, Elasticsearch, Kafka, and MQTT protocols without

Lamp Cloud is a multi-tenant SaaS backend framework built on Java and Spring Cloud that provides a complete foundation for building enterprise-grade administration systems. Its core identity centers on supporting multiple tenant isolation strategies—including database-per-tenant, schema-per-tenant, and shared-table modes—that can be switched without altering business code, alongside a role-based access control system enforced at the gateway layer across all microservices. The framework distinguishes itself through comprehensive tenant lifecycle management tools that allow creating, configurin

Weblate is an open-source web-based translation management system that provides a collaborative platform for teams to review, suggest, and approve translations in real time. It functions as a continuous localization platform, automatically synchronizing translations with source code changes in version control repositories, and can be deployed either as a self-hosted server or through a managed cloud hosting service. The system integrates directly with Git hosting platforms like GitHub, GitLab, and Bitbucket, storing all translations in version control with individual translator attribution re

Tortoise ORM is an asynchronous object-relational mapper for Python that mirrors Django's model and queryset API while running on asyncio. It defines database tables as Python classes with typed fields and supports foreign key, many-to-many, and one-to-one relations, providing a chainable query API for filtering, annotating, grouping, and prefetching related objects without blocking the event loop. The ORM includes a built-in migration engine that detects model changes, generates migration files, and applies or reverts schema changes through a command-line tool. It connects to PostgreSQL, MyS

wger is an open-source web application for fitness tracking, workout planning, and nutrition management. It provides a self-hosted platform where users can design weekly workout routines from a built-in exercise library, log their training progress, and plan daily meals using a food database with automatic nutritional calculations. The application supports multi-user accounts with credential-based login, passkey authentication, and third-party sign-in through OAuth providers. The platform includes a documented REST API that enables programmatic access to workout logs, meal plans, and user dat

CKAN is an open-source data management platform that provides the foundation for building data portals. It supports the full lifecycle of datasets—from creation and organization to publishing, cataloging with faceted search, and interactive data visualization—all through a web interface. The platform is built on a modular architecture that includes a plugin-based extensibility system, a harvesting framework for importing metadata from external sources, and a standardized RESTful JSON API for programmatic access to datasets and metadata. The web interface is rendered using the Jinja2 templatin

Django Silk is a profiling and inspection toolset for Django applications designed to capture SQL queries, HTTP request data, and execution timing for diagnostics. It functions as a performance profiler and debugging middleware that records runtime execution data to provide a comprehensive overview of application behavior. The system includes a database profiler for identifying slow operations through detailed timing data and an HTTP request inspector for reviewing headers, bodies, and network traffic via a web interface. It allows for the reproduction of specific server requests through gene

Athens is a Go module proxy server and dependency cache that provides a persistent storage system for Go dependencies. It acts as a mirror and datastore to ensure reproducible build environments by storing immutable copies of external packages, protecting against upstream deletions or outages. The project distinguishes itself by serving as a secure gateway for private Go module hosting, utilizing authentication tokens, SSH keys, and GitHub Apps to retrieve dependencies from private version control systems. It further enables software dependency compliance through request filtering and checksu

ruvector is a Rust-based vector store and graph database designed for local inference and nearest neighbor searches. It utilizes a vector graph database architecture and a graph neural network index to refine search rankings through structural attention. The system includes a hardware-accelerated quantum circuit simulator for executing state-vector simulations and complex search patterns, alongside a WebAssembly inference engine for running vector search and model execution directly in web browsers. The project employs a cognitive container format that bundles models, data, and a bootable mic

Chonkie ist eine Text-Chunking-Bibliothek, die für Retrieval-Augmented-Generation-Pipelines (RAG) konzipiert wurde. Sie fungiert als semantischer Text-Splitter und RAG-Ingestion-Pipeline und transformiert Rohtext in eingebettete Segmente für die Speicherung in Vektordatenbanken. Das Projekt zeichnet sich durch spezialisierte Splitting-Strategien aus, einschließlich eines AST-basierten Code-Splitters zur Bewahrung logischer Grenzen im Quellcode und eines semantischen Text-Splitters, der Embedding-Modelle verwendet, um Grenzen basierend auf der Bedeutung zu bestimmen. Es bietet zudem einen Vektordatenbank-Ingestor, um die Generierung von Embeddings und deren Export in verschiedene Speicher zu automatisieren. Die Bibliothek deckt ein breites Spektrum an Funktionen ab, einschließlich Dokumenten-Parsing via OCR und Markdown-Extraktion, einer Vielzahl von Splitting-Methoden wie Token-Count und hierarchische Segmentierung sowie Workflow-Orchestrierung durch wiederverwendbare Pipelines. Sie unterstützt eine breite Palette an Vektorspeicher-Integrationen, einschließlich Qdrant, Milvus, Weaviate und Elasticsearch, sowie den Datenexport in JSON- und Hugging-Face-Datensätze. Nutzer können diese Operationen über eine Kommandozeilenschnittstelle ausführen oder das System als containerisierten API-Dienst bereitstellen.

Bazarr ist ein automatisiertes Untertitel-Management-System und Downloader, der darauf ausgelegt ist, Untertitel für Filme und TV-Serien zu finden, zu beziehen und zu synchronisieren. Es fungiert als Begleiter für Medienbibliotheken, der sich über APIs in externe Medienmanager und Server integriert, um fehlende Untertitel aufzuspüren und Bibliotheken auf dem neuesten Stand zu halten. Das Projekt zeichnet sich durch fortschrittliche Medienverarbeitung aus und nutzt neuronale Audio-Transkription, um Untertitel aus Audiospuren zu generieren oder fremdsprachige Dialoge zu übersetzen. Es bietet zudem audiobasierte Synchronisation, um das Timing der Untertitel an den Videoinhalt anzupassen, und kann bestehende Untertitel automatisch aktualisieren, wenn qualitativ hochwertigere Versionen gefunden werden. Das System deckt ein breites Spektrum an Automatisierungsfunktionen ab, einschließlich Sprachprofil-Management, adaptivem Such-Scheduling über mehrere Untertitel-Anbieter hinweg und ereignisgesteuerten Webhook-Triggern. Es unterstützt Datenpersistenz via SQLite oder PostgreSQL und enthält Tools für Path-Mapping-Übersetzungen, um Mediendateien über verschiedene Host-Systeme oder Container hinweg zu lokalisieren. Die Anwendung kann als Hintergrunddienst unter Windows oder über System-Init-Manager auf anderen Plattformen bereitgestellt werden, mit Unterstützung für Reverse-Proxy-Konfigurationen für den Fernzugriff.

Langroid is a multi-agent orchestration framework and tool integration suite designed for building complex AI applications. It serves as a multi-modal integration layer that connects diverse local and remote language models with an agentic retrieval-augmented generation system. The project distinguishes itself through a collaborative message-exchange paradigm, allowing specialized agents to delegate tasks hierarchically and coordinate via structured communication. It features an advanced state management system for conversational AI, including the ability to rewind and prune conversation hist

Gravitino is a federated metadata lake and unified data catalog designed to manage tables, files, and AI models across diverse data sources and cloud storage. It serves as a centralized interface for governing schemas, access controls, and tagging across relational databases, messaging queues, and object stores. The project distinguishes itself by unifying the management of AI assets, such as machine learning models and their version lineages, alongside traditional tabular data. It also implements the Iceberg REST specification to provide a standardized metadata server and proxy for lakehouse