Awesome GitHub RepositoriesCloud Data Access

Polars is a high-performance columnar data processing library designed for efficient analytical workflows. It functions as a structured data library that organizes information into typed columns, utilizing the Apache Arrow memory format to enable zero-copy data sharing and cache-friendly, vectorized operations. The engine is built to handle large-scale tabular datasets, providing both local and distributed analytical runtimes that scale from single-machine environments to multi-node clusters. The project distinguishes itself through a sophisticated lazy query engine that constructs abstract e

Label Studio is a multi-modal data annotation platform designed to create and manage high-quality training datasets for machine learning. It functions as a self-hosted, containerized environment that supports secure, private deployments, including air-gapped configurations. The platform provides a centralized workspace for labeling diverse media types, such as images, text, audio, and time-series data, to support supervised and reinforcement learning workflows. The platform distinguishes itself through deep integration with machine learning backends, enabling active learning loops, automated

This project is a deep learning framework designed for constructing, training, and deploying neural networks across diverse hardware environments. It functions as a high-performance tensor computation library that provides both imperative and symbolic programming interfaces, allowing developers to balance flexible, step-by-step model building with the efficiency of compiled computation graphs. The framework distinguishes itself through a hybrid execution engine that integrates declarative graph compilation with imperative runtime logic. It supports scalable, distributed training across multip

Metaflow is a Python machine learning framework and MLOps workflow orchestrator designed to manage the lifecycle of data pipelines from local prototyping to production. It serves as a distributed compute manager and an experiment tracking system, enabling the creation of reproducible pipelines that transition between development and high-availability production environments. The framework distinguishes itself through an integrated checkpointing system that automatically persists intermediate data artifacts to remote storage, allowing failed runs to be resumed from the last successful step. It

Enso is a visual dataflow programming environment and multi-language data processing engine that compiles Enso, Python, Java, and JavaScript into a unified representation with a shared memory model for zero-overhead inter-language calls. It functions as a self-service data preparation and analysis platform where users can build data pipelines by connecting nodes in a graph, switching between a no-code visual interface and a code view while keeping all changes reviewable. The platform also serves as a cloud data workflow scheduler and API exposer, allowing workflows to run on a timetable or be

OpenDroneMap (ODM) is an open-source aerial drone photogrammetry pipeline that converts 2D images into georeferenced 3D models, orthophotos, point clouds, and digital elevation maps. At its core, the OpenDroneMap Processing Engine orchestrates a complete Structure-from-Motion workflow, from feature extraction through dense reconstruction and tiled output generation, purpose-built for transforming drone-captured imagery into geospatial data products. The toolkit distinguishes itself through GPU-accelerated SIFT feature extraction using CUDA-capable NVIDIA graphics cards, roughly doubling proce

Goofys هو بوابة تخزين كائنية سحابية متوافقة مع POSIX، تقوم بعرض حاويات التخزين البعيدة كأدلة محلية في نظام الملفات. ينفذ المشروع نظام ملفات في مساحة المستخدم (user-space) يربط خدمات التخزين المتوافقة مع S3 وAzure Blob بنقاط تثبيت محلية، مما يسمح بالوصول إلى الكائنات البعيدة عبر عمليات نظام الملفات القياسية. يوفر المشروع إمكانيات تثبيت محددة لحسابات Amazon S3 وAzure Blob Storage وAzure Data Lake. ويستخدم تنفيذًا يعتمد على FUSE لربط تخزين الكائنات السحابي بنواة نظام التشغيل. يتضمن النظام تحسينات في الأداء مثل التخزين المؤقت للقراءة محليًا لتقليل زمن الوصول، وجلب النطاقات المتزامن لتحسين تنزيل الكائنات الكبيرة. كما يحاكي هياكل المجلدات الهرمية عن طريق تحليل بادئات مفاتيح الكائنات لتوفير محاكاة للأدلة.

This project is a web application security standard and vulnerability framework. It provides a comprehensive list of the most critical security risks facing web applications, paired with technical guidance and a structured methodology for identifying and mitigating these flaws. The framework functions as a secure coding guide and a risk assessment methodology, offering a standardized approach to prioritizing vulnerabilities based on their potential impact and likelihood of exploitation. It defines architectural patterns and technical recommendations to help developers implement defense in dep

Leafmap is a Python geospatial visualization library designed for creating interactive maps and performing geospatial analysis within Jupyter environments. It provides a comprehensive set of tools for building interactive map interfaces, browsing and visualizing SpatioTemporal Asset Catalog items, and connecting to PostGIS databases for spatial data rendering. The project distinguishes itself through a backend-agnostic rendering system that allows users to switch between different mapping engines while maintaining a consistent API. It features specialized capabilities for Cloud Optimized GeoT

mimic-code is a clinical data analysis framework and toolset for processing deidentified electronic health records and intensive care unit data. It provides a healthcare SQL query library and a processing tool to transform raw health records into formats suitable for longitudinal analysis and machine learning. The project features a medical research notebook environment that integrates with cloud-hosted datasets, allowing for remote querying and analysis. It includes a DICOM imaging pipeline to retrieve chest radiographs and link medical imaging with structured clinical metadata. The framewo