Awesome GitHub RepositoriesCloud Data Access

Polars is a high-performance columnar data processing library designed for efficient analytical workflows. It functions as a structured data library that organizes information into typed columns, utilizing the Apache Arrow memory format to enable zero-copy data sharing and cache-friendly, vectorized operations. The engine is built to handle large-scale tabular datasets, providing both local and distributed analytical runtimes that scale from single-machine environments to multi-node clusters. The project distinguishes itself through a sophisticated lazy query engine that constructs abstract e

Label Studio is a multi-modal data annotation platform designed to create and manage high-quality training datasets for machine learning. It functions as a self-hosted, containerized environment that supports secure, private deployments, including air-gapped configurations. The platform provides a centralized workspace for labeling diverse media types, such as images, text, audio, and time-series data, to support supervised and reinforcement learning workflows. The platform distinguishes itself through deep integration with machine learning backends, enabling active learning loops, automated

This project is a deep learning framework designed for constructing, training, and deploying neural networks across diverse hardware environments. It functions as a high-performance tensor computation library that provides both imperative and symbolic programming interfaces, allowing developers to balance flexible, step-by-step model building with the efficiency of compiled computation graphs. The framework distinguishes itself through a hybrid execution engine that integrates declarative graph compilation with imperative runtime logic. It supports scalable, distributed training across multip

Metaflow is a Python machine learning framework and MLOps workflow orchestrator designed to manage the lifecycle of data pipelines from local prototyping to production. It serves as a distributed compute manager and an experiment tracking system, enabling the creation of reproducible pipelines that transition between development and high-availability production environments. The framework distinguishes itself through an integrated checkpointing system that automatically persists intermediate data artifacts to remote storage, allowing failed runs to be resumed from the last successful step. It

Enso is a visual dataflow programming environment and multi-language data processing engine that compiles Enso, Python, Java, and JavaScript into a unified representation with a shared memory model for zero-overhead inter-language calls. It functions as a self-service data preparation and analysis platform where users can build data pipelines by connecting nodes in a graph, switching between a no-code visual interface and a code view while keeping all changes reviewable. The platform also serves as a cloud data workflow scheduler and API exposer, allowing workflows to run on a timetable or be

OpenDroneMap (ODM) is an open-source aerial drone photogrammetry pipeline that converts 2D images into georeferenced 3D models, orthophotos, point clouds, and digital elevation maps. At its core, the OpenDroneMap Processing Engine orchestrates a complete Structure-from-Motion workflow, from feature extraction through dense reconstruction and tiled output generation, purpose-built for transforming drone-captured imagery into geospatial data products. The toolkit distinguishes itself through GPU-accelerated SIFT feature extraction using CUDA-capable NVIDIA graphics cards, roughly doubling proce

Goofys 是一个兼容 POSIX 的云对象存储网关，可将远程存储桶挂载为本地系统目录。它实现了一个用户空间文件系统，将兼容 S3 和 Azure Blob 的存储服务映射到本地挂载点，从而允许通过标准文件系统操作访问远程对象。 该项目为 Amazon S3、Azure Blob Storage 和 Azure Data Lake 账户提供了特定的挂载功能。它利用基于 FUSE 的实现将云对象存储与操作系统内核进行对接。 该系统包含性能优化功能，例如本地读取缓存以减少延迟，以及并发范围请求获取以优化大对象的下载。它通过解析对象键前缀来模拟分层文件夹结构，从而提供目录仿真。

This project is a web application security standard and vulnerability framework. It provides a comprehensive list of the most critical security risks facing web applications, paired with technical guidance and a structured methodology for identifying and mitigating these flaws. The framework functions as a secure coding guide and a risk assessment methodology, offering a standardized approach to prioritizing vulnerabilities based on their potential impact and likelihood of exploitation. It defines architectural patterns and technical recommendations to help developers implement defense in dep

Leafmap is a Python geospatial visualization library designed for creating interactive maps and performing geospatial analysis within Jupyter environments. It provides a comprehensive set of tools for building interactive map interfaces, browsing and visualizing SpatioTemporal Asset Catalog items, and connecting to PostGIS databases for spatial data rendering. The project distinguishes itself through a backend-agnostic rendering system that allows users to switch between different mapping engines while maintaining a consistent API. It features specialized capabilities for Cloud Optimized GeoT

mimic-code is a clinical data analysis framework and toolset for processing deidentified electronic health records and intensive care unit data. It provides a healthcare SQL query library and a processing tool to transform raw health records into formats suitable for longitudinal analysis and machine learning. The project features a medical research notebook environment that integrates with cloud-hosted datasets, allowing for remote querying and analysis. It includes a DICOM imaging pipeline to retrieve chest radiographs and link medical imaging with structured clinical metadata. The framewo