Awesome GitHub RepositoriesDynamic Schema Storage

ClickHouse is a high-performance, columnar analytical database designed for real-time query execution and large-scale data aggregation. It functions as a distributed data warehouse capable of processing petabytes of information, while also providing an embedded engine that integrates directly into applications for native query capabilities without external dependencies. The system is built to handle high-throughput ingestion and complex analytical workloads, delivering millisecond-level latency for interactive dashboards and operational monitoring. The platform distinguishes itself through ad

dbt-core is a command-line framework for transforming data within a warehouse using modular SQL and version control. It functions as a data transformation engine that enables users to define data structures and business logic through declarative configuration files, which the system then compiles into executable code. By managing complex data dependencies through a directed acyclic graph, it ensures that transformation tasks execute in the correct order while maintaining a manifest-driven state to track lineage and execution history. The project distinguishes itself through an adapter-based d

This project is a collection of utilities designed for machine learning experiment tracking, data versioning, and the observability of large language model applications. It provides a client for recording hyperparameters and metrics during training to visualize performance trends and compare different model versions. The tool includes a model evaluation framework that uses custom scorers and automated judges to assess the quality of generated text outputs. It also provides observability tools to monitor and debug the execution flow and runtime behavior of language model applications. The sys

This project is a MongoDB Eloquent ORM and NoSQL query builder for the Laravel framework. It provides an active record implementation that maps MongoDB collections and documents to programmable models for data manipulation. The system enables schemaless data management, allowing applications to handle dynamic data structures without the need for rigid database migrations or predefined tables. It integrates MongoDB into Laravel applications to store and retrieve flexible document data using standard PHP patterns. The library covers document store querying and Eloquent model mapping, utilizing

Firetable ist eine Backend-Administrationsplattform und ein Low-Code-Orchestrator für Firebase-Datenbanken. Es bietet eine tabellenkalkulationsähnliche Oberfläche zur Verwaltung von Cloud-Datenbankeinträgen, zur Definition komplexer Datenfelder und zur Steuerung von Benutzerzugriffsrechten. Das Projekt fungiert als No-Code-Automatisierungsplattform und Cloud-Function-Orchestrator, der die Bereitstellung und Verwaltung serverloser Logik über eine visuelle Browser-Oberfläche ermöglicht. Es erlaubt die Erstellung von Backend-Workflows und externen API-Integrationen, die durch spezifische Datenänderungen auf Feldebene ausgelöst werden. Die Plattform deckt breite Funktionsbereiche ab, darunter rollenbasierte Zugriffskontrolle für Tabellen- und Feldsicherheit, dynamische Schema-Auflösung für benutzerdefinierte Datentypen sowie Tools für den Massenimport und -export von Daten. Zudem integriert sie Echtzeit-Datenabrufe von externen Web-Endpunkten, um Datenbankzellen zu befüllen.

Lance is a columnar data format and storage layer designed for high-performance random access and the persistence of multimodal data. It functions as a vector database storage system, a multimodal data store, and a versioned dataset manager. The project distinguishes itself as a hybrid search engine that combines vector similarity search and full-text indexing on a single dataset. It provides unified storage for diverse data types including images, audio, and video, utilizing a system that lazy-loads large binary objects only when requested. The system manages dataset evolution through schem

Pinot is a distributed, columnar analytical database designed for high-concurrency, low-latency query processing. It functions as a real-time OLAP datastore, enabling interactive, user-facing analytics by ingesting and querying massive datasets from both streaming and batch sources. The system architecture relies on a centralized controller for cluster coordination and a distributed segment-based storage model to ensure horizontal scalability. The platform distinguishes itself through a hybrid ingestion pipeline that unifies real-time event streams and historical batch data into a single quer

zvec is an embedded vector database engine and indexing library designed for high-dimensional similarity search. It functions as a hybrid search engine and a retrieval-augmented generation knowledge base, allowing for the storage and retrieval of dense and sparse vectors. The system is distinguished by its hybrid retrieval pipeline, which fuses vector similarity, full-text keyword matching, and scalar metadata filtering into single query operations. It supports a plugin-based model integration system for registering custom embedding models and rerankers, as well as language bindings for nativ

Jitsu is an open-source customer data platform designed to orchestrate event data pipelines. It captures, transforms, and routes behavioral data from web and server sources into data warehouses and analytics tools, providing a unified infrastructure for managing event streams. The platform distinguishes itself through its focus on self-hosted, containerized operations that grant users full control over their data security and privacy. It features a robust identity resolution engine that stitches disparate user identifiers into persistent profiles across sessions and devices, alongside program

Rudder Server ist eine Customer Data Platform (CDP) und Event-Routing-Pipeline, die darauf ausgelegt ist, Kundendaten zu sammeln, zu transformieren und von verschiedenen Quellen an Data Warehouses und Business-Tools weiterzuleiten. Es fungiert als Customer-Identity-Resolver, der Identifikatoren aus mehreren Quellen verknüpft, um einen einheitlichen Identitätsgraphen und umfassende verhaltensbasierte Kundenprofile zu erstellen. Das System zeichnet sich durch Reverse-ETL-Funktionen aus, die verarbeitete Kundensegmente und Zielgruppen aus Data Warehouses zurück in operative Drittanbieteranwendungen pushen. Es bietet zudem eine containerisierte Datenebene für Kubernetes-Deployments, was die Verwaltung der Dateninfrastruktur als Code ermöglicht. Die Plattform deckt eine breite Palette von Datenmanagement-Funktionen ab, einschließlich Echtzeit-Event-Transformation, Schema-Validierung via Datenkatalogen und Privacy-Governance. Dazu gehören Tools zur Verwaltung der Benutzereinwilligung, zur Durchsetzung der Datenresidenz innerhalb spezifischer geografischer Regionen und zur Maskierung personenbezogener Daten während der Übertragung. Installation und Deployment der Datenebenen-Komponenten werden mittels Helm-Charts verwaltet.

PagesCMS is a headless, Git-based content management system that uses a GitHub repository as its primary database. It provides a web interface for authoring and organizing structured files and media assets, transforming flat files in a repository into a structured content management environment. The system automates repository access and content updates through GitHub App integration and manifests. It features a schema-driven approach to map repository files and folders to specific editor fields, allowing for the management of static site content without a separate database. Capability areas