Awesome GitHub RepositoriesCloud-Local Data Interfaces

DeepLake is AI data infrastructure consisting of a multimodal data lake, a hybrid search engine, and a serverless vector database. It provides a PostgreSQL-based AI data runtime that combines multimodal storage with streaming pipelines to load and shuffle datasets from cloud storage directly into deep learning training pipelines. The system utilizes lazy indexing to store and slice images, audio, and video without loading entire files into memory. It enables retrieval-augmented generation by persisting high-dimensional embeddings in a serverless vector store and implementing hybrid search tha

bypy is a Python client for interacting with Baidu Yun cloud storage via its official API. It functions as both a library for programmatic integration and a command-line manager for organizing and listing files within a cloud account. The project features a multi-process transfer tool that utilizes parallel processing to accelerate high-volume uploads and downloads. It provides bidirectional directory synchronization to align content between local file systems and remote cloud storage by identifying missing or outdated files. The tool covers recursive file transfers, automated cloud data bac

KubeEdge is a distributed edge computing framework that extends Kubernetes to manage containerized workloads and hardware devices at the edge. It functions as a Kubernetes edge orchestration system, allowing the deployment and management of applications across distributed edge nodes using native Kubernetes APIs and workflows. The project distinguishes itself through a specialized focus on IoT integration and node autonomy. It employs digital-twin state modeling to represent physical hardware devices as virtual objects, utilizing an MQTT-based messaging bus for communication with heterogeneous

Flyte is a Kubernetes-based machine learning orchestrator and containerized pipeline manager designed for coordinating AI workflows and data pipelines. It functions as an engine for defining and executing resilient pipelines, utilizing a data lineage tracker to maintain immutable execution states and ensure reproducible outputs. The platform distinguishes itself by packaging individual tasks into separate containers to ensure dependency isolation and environment consistency. It provides specialized capabilities for machine learning, including the transformation of trained models into scalable

Falcon هو عميل SQL ومدير قواعد بيانات متعدد المنصات يوفر واجهة موحدة لتنفيذ الاستعلامات عبر أنظمة قواعد بيانات متعددة. يعمل كتطبيق سطح مكتب لنظامي Windows و Mac، مما يسمح للمستخدمين بإدارة البيانات عبر بيئات قواعد بيانات متنوعة من خلال محرر استعلام واحد. تعمل الأداة كاستوديو بيانات متزامن سحابياً، مما يسد الفجوة بين نتائج قواعد البيانات المحلية والتخزين السحابي البعيد. وهذا يتيح نقل بيانات الاستعلام المعالجة إلى بيئة سحابية للتخزين الدائم والتحليل التعاوني. إلى جانب تنفيذ الاستعلامات، يتضمن التطبيق أداة تصور تحول نتائج SQL إلى مخططات تفاعلية لتحليل البيانات. وتدعم هذه الإمكانات نظام اتصال يعتمد على التعريفات (drivers) وآلية تخزين مؤقت من جانب العميل لإدارة مجموعات النتائج.