15 个仓库
Collaborative repositories for indexing and sharing structured data sources.
Distinct from Open Source Software: Distinct from general open-source software: focuses on the collaborative maintenance of data-centric registries rather than code libraries.
Explore 15 awesome GitHub repositories matching development tools & productivity · Data Catalogs. Refine with filters or upvote what's useful.
这是一个由社区策划的开源软件目录,专为在私有服务器环境和家庭实验室中部署而设计。它作为发现主流云服务独立自托管替代方案的综合资源,使用户能够保持对数字基础设施的完全数据所有权和控制权。 该目录通过层级分类法构建,将庞大的应用程序集合组织成逻辑类别,范围从媒体管理和数据分析到私有通信和团队生产力工具。它通过协作同行评审流程脱颖而出,社区成员验证每个提交的质量和相关性,以确保目录保持准确和可靠。 该项目涵盖了广泛的能力领域,包括基础设施自动化、基于容器的服务部署和声明式配置管理。这些工具协助用户维护可复现的服务器环境,并管理私有硬件上的复杂服务依赖。 该目录作为版本控制仓库进行维护,确保所有更新和社区驱动的变更都是可追踪且透明的。
Stores and organizes recorded trail paths and associated metadata into a searchable database for outdoor enthusiasts.
Dagster is a data orchestration platform designed to manage the entire lifecycle of data assets through declarative modeling and version-controlled code. It functions as a workflow engine that treats data assets as first-class primitives, allowing teams to define, schedule, and monitor complex pipelines while maintaining clear visibility into lineage, dependencies, and data quality. The platform distinguishes itself by using a code-as-configuration framework that enables standard software engineering practices, such as unit testing and local mocking, to be applied directly to data workflows.
Maintains a centralized view of data assets, workflows, and lineage to help teams discover and reuse components.
The AWS Cloud Development Kit is an infrastructure-as-code framework that enables developers to define and provision cloud resources using familiar programming languages. By utilizing construct-based synthesis, it translates high-level, object-oriented code into declarative templates, allowing for the automated management of complex cloud environments through a centralized, code-driven control plane. The framework distinguishes itself through its ability to model infrastructure as a dependency-aware resource graph, ensuring that components are provisioned and updated in the correct order. It
Registers data moved into storage within a central metadata repository to simplify discovery and access for analytics and machine learning tools.
DataHub is a metadata management platform designed to unify technical, operational, and business context across diverse data ecosystems. By utilizing a graph-based metadata model and an event-driven ingestion architecture, it creates a centralized source of truth that maps complex data relationships, lineage, and ownership. This foundational framework enables organizations to maintain a synchronized view of their data landscape, supporting both human-led discovery and automated data operations. The platform distinguishes itself through its focus on grounding artificial intelligence and autono
Maintains a centralized repository of metadata to provide transparency into data sources and usage patterns.
InsForge is a backend-as-a-service platform that provides an integrated suite of tools for managing relational databases, identity provision, object storage, and serverless compute. It functions as an open-source identity provider and a PostgreSQL database manager featuring integrated vector storage and row-level security. The platform serves as an LLM orchestration gateway, offering a unified endpoint to route requests across various AI providers through an OpenAI-compatible interface. It enables AI-driven application generation and connects AI agents to backend resources using a standardize
Maintains a standardized catalog of available AI models to identify their capabilities and modalities.
Clair is a container image vulnerability scanner and security analyzer. It performs static analysis of container images by matching package contents against vulnerability databases to identify security risks across different package formats and architectures. The project functions as both an image indexer and a vulnerability database manager. It processes container layers into intermediate representations to enable fast security lookups and synchronizes security metadata from multiple external sources to maintain a local registry. Capability areas include continuous security monitoring, whic
Periodically synchronizes security metadata from remote vulnerability databases to maintain a local searchable registry.
Delta is a lakehouse table format that brings ACID transactions and data warehouse consistency to large scale data lakes on cloud object storage. It serves as an ACID transaction manager, coordinating atomic commits and serializable isolation for concurrent reads and writes across distributed compute engines. The project provides a multi-engine interoperability layer that uses format translation to allow diverse SQL engines and processing frameworks to read and write the same tables. It functions as a data versioning system, utilizing a transaction log to enable time travel, historical snapsh
Generates compatible metadata for external table formats at commit time to enable cross-engine interoperability.
Higress is an AI API gateway and cloud-native traffic manager that functions as a Kubernetes ingress controller. It provides a centralized system for routing, securing, and optimizing traffic directed toward large language models, AI agents, and microservice architectures. The project distinguishes itself through deep AI orchestration, including the ability to host and manage Model Context Protocol servers that transform REST APIs into tools for AI agents. It features specialized AI infrastructure for model request proxying, protocol translation across multiple providers, and semantic-based c
Distributes models and agents through a centralized artifact catalog featuring versioning and grayscale releases.
CloudQuery is a cloud infrastructure ETL tool and multi-cloud data pipeline designed to collect, synchronize, and normalize resource metadata from various cloud providers and SaaS platforms. It functions as a centralized asset inventory manager and security posture manager, extracting configuration and state data into relational databases, data lakes, or data warehouses. The system distinguishes itself by transforming complex, nested cloud API responses into flat relational tables, enabling the use of standard SQL for asset querying and analysis. It employs a modular plugin system for data ex
Automates the synchronization of resource configurations from multiple cloud providers into a centralized destination.
NextFaster is a Next.js e-commerce storefront that uses artificial intelligence to generate a complete product catalog, including categories, descriptions, and images. Product images are stored in cloud object storage and served directly via CDN, offloading delivery from the origin server. Pages are delivered with pre-rendered shells from the edge, with dynamic content streamed in afterward for fast initial interactivity. All write operations—such as orders and catalog updates—are handled through server-side functions for data consistency and security. The project differentiates itself by com
Generates a complete product catalog with categories, descriptions, and images using AI.
BilibiliSponsorBlock is a content filtering system and API server designed to identify and remove sponsored segments and filler content from Bilibili video playback. It utilizes a crowdsourced segment database where users contribute and vote on timestamps to create a shared repository of skippable video sections. The project features a video metadata synchronizer that links equivalent videos across different platforms, allowing skip markers and timing data to be shared between mirrored content. It implements a reputation-based permission system to manage submissions and edits, alongside a pri
Links videos across different platforms to synchronize skip markers and timing data.
该项目是一个 AWS pandas 集成库和数据流水线框架,旨在简化本地内存与 AWS 存储及分析服务之间的数据移动和转换。它作为一个云数据湖工具包和存储文件管理器,允许用户在各种云环境中读取、写入和转换结构化数据。 该库作为分布式计算编排器脱颖而出,能够在 EMR 等环境中管理集群,以处理超出单机内存限制的数据集。它还提供用于管理向量索引和在云存储桶内执行相似度搜索的专门功能。 其更广泛的功能面涵盖了针对 DynamoDB、RDS 和 Timestream 等服务的云数据库 ETL,以及通过 AWS Glue 进行的云数据目录管理。它支持通过 Athena 和 Redshift 进行无服务器数据分析,并提供用于管理 S3 对象、在 OpenSearch 中索引文档以及分析 CloudWatch 日志的实用程序。
Automatically synchronizes table metadata with central catalogs when writing data frames to cloud storage.
Wanderer is a self-hosted trail database and federated social network for storing GPS tracks and hiking routes. It functions as a GPX route manager and geographic route planner that allows users to catalog outdoor explorations in a private database. The system utilizes the ActivityPub protocol to enable decentralized federation, allowing users to follow others and share activities across independent server instances. This social layer includes collaborative route exchange, engagement management for likes and comments, and granular content visibility controls. The platform provides tools for
Provides a self-hosted database for cataloging and searching outdoor travel tracks and hiking route metadata.
This project is a self-hosted media server for organizing, streaming, and tracking anime and manga collections. It functions as a BitTorrent streaming client that allows video content to be played directly from torrents and cloud storage, a manga reader and tracker, and a media processing system using hardware-accelerated transcoding to ensure browser compatibility. The system distinguishes itself through synchronized media viewing, enabling users to host watch parties by coordinating playback in real time across multiple devices. It also features an extensible framework with a JavaScript-bas
Automatically fetches and updates series and episode metadata from external data platforms.
Gravitino is a federated metadata lake and unified data catalog designed to manage tables, files, and AI models across diverse data sources and cloud storage. It serves as a centralized interface for governing schemas, access controls, and tagging across relational databases, messaging queues, and object stores. The project distinguishes itself by unifying the management of AI assets, such as machine learning models and their version lineages, alongside traditional tabular data. It also implements the Iceberg REST specification to provide a standardized metadata server and proxy for lakehouse
Maintains structured catalogs for AI-specific artifacts, including model metadata and governance.