Awesome GitHub RepositoriesPostgreSQL Vector Stores

This project is a privacy-first backend service designed to facilitate retrieval-augmented generation by processing local documents into searchable vector representations. It provides a modular architecture that allows users to ingest diverse file formats, manage document metadata, and perform semantic searches to provide context-aware responses for chat and completion requests. The system distinguishes itself through a database-agnostic abstraction layer that supports various storage backends, ranging from local disk storage to enterprise-grade vector databases. It offers flexible deployment

Vector is a high-performance observability data pipeline designed to collect, transform, and route logs, metrics, and traces across distributed infrastructure. It functions as a modular engine that decouples data ingestion from processing and transmission, utilizing a component-based architecture to connect diverse sources to multiple destinations. The project distinguishes itself through a focus on reliability and flow control. It implements backpressure-aware data movement to prevent data loss during traffic spikes and utilizes disk-backed event buffering to ensure durability during network

This project is a retrieval-augmented generation pipeline designed for building custom ChatGPT plugins that allow language models to query private or professional documents. It implements a full retrieval workflow, from processing and indexing document chunks to retrieving relevant context for natural language queries. The system distinguishes itself through a hybrid retrieval approach that combines dense vector embeddings with sparse keyword matching, further refined by a two-stage semantic re-ranking process. It includes specialized data privacy tools for screening personally identifiable i

Hub is a multimodal AI data lake and vector database designed for storing and querying embeddings, text, audio, and images. It functions as a dataset version control system and a machine learning data streaming engine to support large-scale model training. The system utilizes a serverless PostgreSQL vector store to index high-dimensional embeddings for semantic search. It provides a visual interface for inspecting multimodal datasets and viewing annotations such as bounding boxes and masks. The platform handles cloud-agnostic storage synchronization and implements lazy, compressed data strea

Arize Phoenix is an LLM observability platform and evaluation framework designed to capture execution traces and monitor large language model applications. It serves as a prompt management system for versioning and testing templates, and as a self-hosted AI operations infrastructure for managing telemetry and experiments. The platform differentiates itself through a specialized embedding visualization tool used to detect data drift and optimize vector search. It provides a comprehensive evaluation suite that utilizes judge-based evaluators and ground-truth datasets to score model outputs, and

pgloader is a command-line tool that automates the migration of data and schema from various source databases and file formats into PostgreSQL. It combines schema discovery, parallel data pipelines, and type casting into a single, declarative workflow, using PostgreSQL's COPY protocol for high-throughput bulk loading. The tool distinguishes itself by compiling a dedicated command language into concurrent reader-writer pipelines that handle schema introspection, data transformation, and error-resilient batch processing. It supports migrating entire databases from MySQL, MS SQL, SQLite, and Pos

GreptimeDB is a distributed, open-source time-series database built for unified observability. It stores and queries metrics, logs, and traces together in a single columnar engine, supporting both SQL and PromQL for analysis. The database is designed as a Kubernetes-native operator with a decoupled compute and storage architecture, enabling horizontal scaling and multi-region deployment. What distinguishes GreptimeDB is its role as a multi-protocol ingestion gateway, accepting data through OpenTelemetry, Prometheus Remote Write, InfluxDB, Loki, Elasticsearch, Kafka, and MQTT protocols without

Lamp Cloud is a multi-tenant SaaS backend framework built on Java and Spring Cloud that provides a complete foundation for building enterprise-grade administration systems. Its core identity centers on supporting multiple tenant isolation strategies—including database-per-tenant, schema-per-tenant, and shared-table modes—that can be switched without altering business code, alongside a role-based access control system enforced at the gateway layer across all microservices. The framework distinguishes itself through comprehensive tenant lifecycle management tools that allow creating, configurin

Weblate is an open-source web-based translation management system that provides a collaborative platform for teams to review, suggest, and approve translations in real time. It functions as a continuous localization platform, automatically synchronizing translations with source code changes in version control repositories, and can be deployed either as a self-hosted server or through a managed cloud hosting service. The system integrates directly with Git hosting platforms like GitHub, GitLab, and Bitbucket, storing all translations in version control with individual translator attribution re

Tortoise ORM is an asynchronous object-relational mapper for Python that mirrors Django's model and queryset API while running on asyncio. It defines database tables as Python classes with typed fields and supports foreign key, many-to-many, and one-to-one relations, providing a chainable query API for filtering, annotating, grouping, and prefetching related objects without blocking the event loop. The ORM includes a built-in migration engine that detects model changes, generates migration files, and applies or reverts schema changes through a command-line tool. It connects to PostgreSQL, MyS

wger is an open-source web application for fitness tracking, workout planning, and nutrition management. It provides a self-hosted platform where users can design weekly workout routines from a built-in exercise library, log their training progress, and plan daily meals using a food database with automatic nutritional calculations. The application supports multi-user accounts with credential-based login, passkey authentication, and third-party sign-in through OAuth providers. The platform includes a documented REST API that enables programmatic access to workout logs, meal plans, and user dat

CKAN is an open-source data management platform that provides the foundation for building data portals. It supports the full lifecycle of datasets—from creation and organization to publishing, cataloging with faceted search, and interactive data visualization—all through a web interface. The platform is built on a modular architecture that includes a plugin-based extensibility system, a harvesting framework for importing metadata from external sources, and a standardized RESTful JSON API for programmatic access to datasets and metadata. The web interface is rendered using the Jinja2 templatin

Django Silk is a profiling and inspection toolset for Django applications designed to capture SQL queries, HTTP request data, and execution timing for diagnostics. It functions as a performance profiler and debugging middleware that records runtime execution data to provide a comprehensive overview of application behavior. The system includes a database profiler for identifying slow operations through detailed timing data and an HTTP request inspector for reviewing headers, bodies, and network traffic via a web interface. It allows for the reproduction of specific server requests through gene

Athens is a Go module proxy server and dependency cache that provides a persistent storage system for Go dependencies. It acts as a mirror and datastore to ensure reproducible build environments by storing immutable copies of external packages, protecting against upstream deletions or outages. The project distinguishes itself by serving as a secure gateway for private Go module hosting, utilizing authentication tokens, SSH keys, and GitHub Apps to retrieve dependencies from private version control systems. It further enables software dependency compliance through request filtering and checksu

ruvector هو مخزن متجهات وقاعدة بيانات رسومية مبني بلغة Rust، مصمم للاستدلال المحلي وعمليات البحث عن أقرب جار. يستخدم بنية قاعدة بيانات المتجهات الرسومية وفهرس الشبكات العصبية الرسومية لتحسين ترتيب نتائج البحث من خلال الانتباه الهيكلي. يتضمن النظام محاكي دوائر كمومية معزز بالأجهزة لتنفيذ محاكاة متجهات الحالة وأنماط البحث المعقدة، إلى جانب محرك استدلال WebAssembly لتشغيل البحث عن المتجهات وتنفيذ النماذج مباشرة داخل متصفحات الويب. يستخدم المشروع تنسيق حاوية معرفي يجمع النماذج والبيانات ونواة صغيرة قابلة للإقلاع في ملف ثنائي واحد للنشر. كما يتميز بأدوات متخصصة لتهيئة النماذج، بما في ذلك طريقة لدمج الأوزان لمنع النسيان الكارثي وآلية محول خفيفة الوزن لتكييف الأوزان بشكل فوري. يغطي النظام نطاقاً واسعاً من القدرات بما في ذلك البحث عن المتجهات المعزز بالأجهزة، والاستعلام عن العلاقات الرسومية، وتحليل المستندات العلمية لاستخراج LaTeX و MathML. كما يوفر تسلسل شهادات التشفير للتحقق من تغييرات البيانات، ومزامنة البيانات الوصفية القائمة على Raft لضمان التوافر العالي، وضغط البيانات متعدد الدقة لإدارة تكاليف التخزين.

Chonkie هي مكتبة لتقسيم النصوص مصممة لخطوط أنابيب التوليد المعزز بالاسترجاع (RAG). تعمل كمقسم نصوص دلالي وخط أنابيب استيعاب RAG، حيث تحول النص الخام إلى أجزاء مضمنة للتخزين في قواعد بيانات المتجهات. يتميز المشروع باستراتيجيات تقسيم متخصصة، بما في ذلك مقسم كود يعتمد على AST للحفاظ على الحدود المنطقية في الكود المصدري ومقسم نص دلالي يستخدم نماذج التضمين لتحديد الحدود بناءً على المعنى. كما يوفر مستوعباً لقاعدة بيانات المتجهات لأتمتة توليد التضمينات وتصديرها إلى متاجر مختلفة. تغطي المكتبة مجموعة واسعة من الإمكانيات، بما في ذلك تحليل المستندات عبر OCR واستخراج markdown، ومجموعة متنوعة من طرق التقسيم مثل عدد الرموز والتقسيم الهرمي، وتنسيق سير العمل من خلال خطوط أنابيب قابلة لإعادة الاستخدام. تدعم مجموعة واسعة من تكاملات مخزن المتجهات، بما في ذلك Qdrant وMilvus وWeaviate وElasticsearch، بالإضافة إلى تصدير البيانات إلى JSON ومجموعات بيانات Hugging Face. يمكن للمستخدمين تنفيذ هذه العمليات عبر واجهة سطر الأوامر أو نشر النظام كخدمة API محاوية.

Bazarr is an automated subtitle management system and downloader designed to discover, acquire, and synchronize subtitles for movies and TV shows. It functions as a media library companion that integrates with external media managers and servers via APIs to track missing subtitles and ensure libraries are up to date. The project distinguishes itself through advanced media processing, using neural-network audio transcription to generate subtitles from audio tracks or translate foreign dialogue into English. It also features audio-based synchronization to align subtitle timing with video conten

Langroid is a multi-agent orchestration framework and tool integration suite designed for building complex AI applications. It serves as a multi-modal integration layer that connects diverse local and remote language models with an agentic retrieval-augmented generation system. The project distinguishes itself through a collaborative message-exchange paradigm, allowing specialized agents to delegate tasks hierarchically and coordinate via structured communication. It features an advanced state management system for conversational AI, including the ability to rewind and prune conversation hist

Gravitino is a federated metadata lake and unified data catalog designed to manage tables, files, and AI models across diverse data sources and cloud storage. It serves as a centralized interface for governing schemas, access controls, and tagging across relational databases, messaging queues, and object stores. The project distinguishes itself by unifying the management of AI assets, such as machine learning models and their version lineages, alongside traditional tabular data. It also implements the Iceberg REST specification to provide a standardized metadata server and proxy for lakehouse