10 个仓库
Tools for managing and executing data-centric projects and workflows.
Distinguishing note: Focuses on project-level orchestration rather than individual record management.
Explore 10 awesome GitHub repositories matching data & databases · Data Project Management. Refine with filters or upvote what's useful.
This project is an AI agent orchestration platform that provides a visual environment for building, testing, and deploying complex automation workflows. It functions as a low-code development interface where users can chain discrete functional blocks into dependency-aware pipelines to integrate artificial intelligence with external data and services. The platform supports the creation of intelligent conversational agents, automated business processes, and multi-service API orchestrations within a unified workspace. The platform distinguishes itself through its event-driven integration engine,
Manages and executes data projects through programmatic interactions.
Hetty is an HTTP intercepting proxy and web security research toolkit used to capture, inspect, and modify traffic between a browser and a server. It functions as an HTTP request editor for creating and replaying manual requests to test server behavior and as a project-based traffic logger that isolates network logs across different security research engagements. The tool provides a request-response interception loop that pauses outgoing requests and incoming responses in transit, allowing for manual editing or cancellation. It includes a manual request replay engine to construct and transmit
Organizes traffic logs and configurations into project databases to separate security engagements.
Gyroflow is a gyroscope video stabilization software and IMU telemetry processor designed to remove camera shake from video files. It functions as a hardware-accelerated video renderer and lens calibration tool, utilizing embedded or external gyroscope and accelerometer data to perform pixel-level stabilization. The system is distinguished by its ability to integrate with professional non-linear video editing software via plugins, allowing stabilization to be applied directly to timelines without transcoding original footage. It supports diverse telemetry ingestion from camera brands, flight
Embeds stabilization and gyroscope data directly into the host project file to eliminate external sidecar files.
LXD is a unified platform for managing both system containers and virtual machines through a single REST API and command-line interface. It provides a programmatic HTTP interface for controlling the full lifecycle of instances, enabling automation and integration with external tools. The system runs unprivileged containers with per-instance UID/GID mappings, seccomp filters, and AppArmor profiles for kernel-level isolation, while supporting multiple storage backends including directory, Btrfs, LVM, ZFS, Ceph, LINSTOR, and TrueNAS through a unified driver interface. The platform distinguishes
Organizes instances, images, profiles, networks, and storage into separate projects for clean management.
dlt 是一个 Python 数据摄取工具和 ETL 流水线框架,旨在从不同来源获取数据并将其持久化到结构化目标中。它作为一个模式推断引擎,可自动检测数据类型并将嵌套的 JSON 结构扁平化为关系表,将数据从源端移动到数据湖、数据仓库或向量数据库。 该项目通过 AI 驱动的流水线生成脱颖而出,利用大语言模型为 REST API 构建提取代码和连接器。它还支持多模态向量存储和向量数据库的专门填充,以支持 AI 和机器学习应用。 该框架涵盖了广泛的功能,包括自动化模式演进、通过状态跟踪进行增量数据加载,以及通过强制执行数据契约进行数据质量验证。它提供了用于关系数据规范化、加载前后转换的工具,以及针对 SQL 数据库和云对象存储的多种目标适配器。 可观测性通过流水线执行仪表板、列血缘跟踪以及使用基于内容的哈希进行模式版本验证来处理。
Organizes multiple data resources into a single logical source for more efficient loading.
Buildbot is a Python-based continuous integration framework and distributed build orchestrator. It functions as a build automation engine that coordinates the retrieval of source code, the execution of build steps, and the reporting of results through a central controller and a network of remote worker agents. The system is distinguished by a plugin-based extensibility architecture and a master-worker distribution model. It allows for dynamic build modification at runtime and supports a pluggable database backend for persisting system state and historical build data. The project covers a bro
Organizes build workers into logical project groups to streamline the management of the system interface.
Kdenlive is an open-source non-linear video editing suite designed for digital video post-production. Built on the MLT Framework and utilizing KDE Frameworks for its user interface, it provides a multi-track environment for assembling clips, applying transitions, and rendering final video files. The editor distinguishes itself through a comprehensive set of animation and effect tools, including keyframe-based parameter animation with a visual curve editor for fine-tuning transitions. It supports advanced visual modifications such as clip speed remapping, effect region masking, and the integra
Embeds author and copyright information directly into multimedia project files.
Bagel 是一个原生的网络调试和检查系统,旨在捕获和监控来自 iOS 模拟器和物理硬件的 HTTP 流量。它作为一个 HTTP 流量检查器和基于设备的监控器,按项目和设备组织捕获的网络请求。 该工具支持直接在操作系统内拦截网络请求,无需手动配置代理或安装自定义安全证书。 它提供了移动流量分析和网络调试功能,包括将拦截的请求分组到逻辑项目桶中,并按唯一设备标识符对日志进行分段的能力。
Groups intercepted network requests into logical project buckets for easier filtering and historical analysis.
Polyaxon is a Kubernetes-native machine learning orchestration platform and MLOps pipeline orchestrator. It serves as a control plane for managing distributed deep learning workloads, automated machine learning pipelines, and experiment tracking. The platform distinguishes itself through specialized services for distributed training management, including MPI-based coordination for PyTorch and TensorFlow. It provides an automated hyperparameter optimization service utilizing Bayesian, random, and grid search algorithms, alongside managed interactive AI workspaces for launching Jupyter notebook
Groups resources and users into projects or separate deployments to isolate workloads via Kubernetes namespaces.
DevLake is a DevOps data platform and analytics tool designed to orchestrate data pipelines that ingest, transform, and sync metadata from external development tools into a unified database. It functions as a system for collecting and normalizing data from source control, CI/CD pipelines, and issue trackers into a standardized schema to enable consistent software delivery analytics. The platform distinguishes itself by transforming tool-specific data into a common domain model, allowing for the calculation of engineering metrics via SQL. It provides specialized frameworks for measuring DORA m
Organizes data scopes from multiple tools into single logical projects for unified performance analysis.