29 مستودعات
Strategies for controlling how and when related data is fetched from a database to optimize memory and performance.
Distinct from Data Loading: Candidates focus on web page load or code loading, not database entity loading strategies (e.g., eager/lazy).
Explore 29 awesome GitHub repositories matching data & databases · Data Loading Optimizations. Refine with filters or upvote what's useful.
SwitchyOmega is a browser proxy manager and extension that controls network traffic routing and manages proxy configurations through a graphical interface. It functions as a proxy routing orchestrator and a PAC script generator, converting user-defined rules into JavaScript auto-config scripts for automated network routing. The project enables users to organize network configurations into distinct profiles, allowing for the switching of proxying logic based on different environments. It automates the selection of proxy servers by matching requested URLs against predefined patterns to avoid ma
Uses the browser's native proxy API to intercept requests and apply routing logic.
ivideo is a cross-platform media client and video aggregator that consolidates content from multiple streaming platforms into a single unified interface. It functions as a webview content wrapper that embeds third-party video pages within a native browser component for use across Windows, Mac, and Linux. The project operates as a premium content unlocker by intercepting network requests and modifying headers to bypass subscription requirements and access restricted resources. It further enables regional media access through user-agent spoofing, allowing users to view video streams from specif
Intercepts and modifies browser-level network requests to bypass subscription requirements and access restricted media resources.
Mikro-ORM is a TypeScript-based object-relational mapping system that provides a unified persistence layer for Node.js applications. It translates TypeScript entities into relational or document-based database schemas, supporting a variety of engines including PostgreSQL, MySQL, MariaDB, MS SQL Server, SQLite, and MongoDB. The project implements the data mapper pattern to decouple in-memory domain models from the database persistence layer. It utilizes a unit of work pattern to track entity changes in memory and commit them in a single coordinated database transaction. The library covers com
Offers loading strategies and partial field fetching to reduce memory usage and improve response times.
Delta is a lakehouse table format that brings ACID transactions and data warehouse consistency to large scale data lakes on cloud object storage. It serves as an ACID transaction manager, coordinating atomic commits and serializable isolation for concurrent reads and writes across distributed compute engines. The project provides a multi-engine interoperability layer that uses format translation to allow diverse SQL engines and processing frameworks to read and write the same tables. It functions as a data versioning system, utilizing a transaction log to enable time travel, historical snapsh
Loads log metadata incrementally when switching versions to reduce latency during time travel.
This project is a comprehensive collection of educational examples and reference implementations for building vision and language models using PyTorch. It serves as a deep learning tutorial covering the end-to-end process of developing neural networks, from initial architecture definition to final production deployment. The repository provides detailed guides on implementing a wide range of domain-specific models, including convolutional neural networks for object detection and segmentation, as well as transformer and recurrent architectures for natural language processing. It emphasizes gene
Provides custom class implementations to define how individual data samples are read from storage.
miniblink49 is a stripped-down Blink browser kernel and embedded web UI framework used to integrate HTML user interfaces into native C++ applications. It functions as a C++ JavaScript interop bridge that maps high-level script calls to low-level native logic and provides a resource-optimized headless browser engine for processing web content without a graphical interface. The project enables custom browser environment control by bypassing cross-origin security policies to allow requests across different domains. It further allows for the interception of network resources to replace remote ass
Intercepts and customizes how the browser kernel loads resources and handles domain restrictions.
Tabulator is an interactive data table library and virtual DOM data grid used to create high-performance tables from JSON or arrays. It functions as a hierarchical data viewer and a spreadsheet interface component, capable of rendering thousands of records efficiently through viewport-based virtualization and progressive loading. The library distinguishes itself by providing a full spreadsheet interface mode with multi-sheet management, cell range selection, and bulk copy-paste capabilities. It supports complex data architectures, including nested data field mapping, expandable tree structure
Allows developers to intercept and modify data loading requests or override the default loading process.
This project is a structured learning curriculum and guide designed to transition a user from a beginner to an advanced level of Emacs mastery over the course of one year. It provides a step-by-step study plan, configuration guides, and a workflow manual to help users navigate the complexities of the editor. The guide emphasizes a specific operational model, focusing on the implementation of modal editing through Vim-style keybindings and the use of hierarchical leader-key mapping for command organization. It details the process of transforming the editor into a full development environment a
Implements interface load optimization by limiting the number of displayed items in mail groups.
mmcv is a foundation library for computer vision based on PyTorch. It provides a comprehensive system for constructing convolutional neural networks, a toolkit for image and video preprocessing, and a collection of high-performance deep learning vision operators. The project is distinguished by its hardware-accelerated kernels for complex operations such as deformable convolutions and region pooling. It features a configuration-driven framework that allows for the dynamic instantiation of network layers and the registration of custom modules without modifying code. The library covers a broad
Implements logic for loading raw image files and corresponding annotations from disk into tensors for training.
هذا المشروع عبارة عن مكون جدول بيانات React مدمج مع shadcn/ui. يوفر مجموعة من شبكات البيانات الجدولية التي تدعم الفرز، والتصفية، والترقيم. يتضمن التنفيذ شبكة بيانات افتراضية وجدول تمرير لانهائي، مما يسمح بعرض مجموعات البيانات الكبيرة عن طريق تحميل السجلات تدريجياً أو عرض العناصر المرئية فقط في إطار العرض. يعمل أيضاً كجدول تعاوني في الوقت الفعلي، مما يتيح لعدة مستخدمين إدارة وتحرير مجموعات البيانات المشتركة في وقت واحد. يغطي المكون إدارة البيانات التفاعلية وتصور البيانات المعقدة. يشمل ذلك القدرة على إدارة اختيار الصفوف من خلال أشرطة الإجراءات السياقية واستخدام التصفية القائمة على تعريف الأعمدة لتنظيم السجلات.
Fetches and displays additional records incrementally as the user scrolls to optimize initial load.
Snorkel is a weak supervision system that enables users to programmatically generate training labels for machine learning models without manual annotation. At its core, it provides a framework for writing labeling functions as Python callables that each vote on data points, and then trains a probabilistic graphical model over these multiple weak supervision sources to estimate latent true labels without any ground truth data. The system automatically learns accuracy and correlation parameters between labeling functions by analyzing observed agreement patterns on unlabeled data, converting lab
Provides a framework for writing Python callables that vote on data points to programmatically label datasets.
GDAL هي مكتبة مترجم مفتوحة المصدر مرخصة بموجب MIT توفر نموذج بيانات مجرداً وموحداً لقراءة وكتابة البيانات الجغرافية المكانية النقطية والمتجهة عبر مئات تنسيقات الملفات. تعمل كمكتبة أساسية لترجمة البيانات الجغرافية المكانية، مما يتيح الوصول إلى تنسيقات بيانات جغرافية مكانية متنوعة من خلال واجهة واحدة متسقة. تكشف المكتبة عن وظائفها الأساسية من خلال أدوات سطر الأوامر التي تسمح للمستخدمين بترجمة وتحويل ومعالجة البيانات الجغرافية المكانية بين التنسيقات. يتعامل محرك تحويل الإحداثيات مع التحويلات بين أنظمة المراجع المكانية، بينما يقوم نظام المكونات الإضافية لبرنامج تشغيل التنسيق بتحميل منطق القراءة والكتابة الخاص بالتنسيق في وقت التشغيل. توفر طبقة نظام الملفات الافتراضية وصولاً موحداً للإدخال/الإخراج عبر الملفات المحلية، وHTTP، والتخزين السحابي، والأرشيفات المضغوطة، ويدير ذاكرة التخزين المؤقت للكتل النقطية التخزين المؤقت للبلاط في الذاكرة لتقليل عمليات الإدخال/الإخراج. تدعم GDAL قراءة وكتابة البيانات الجغرافية المكانية النقطية والمتجهة، مع تكرار ميزات المتجه الذي يبث الميزات بشكل فردي دون تحميل مجموعات البيانات بأكملها في الذاكرة. يتيح المشروع إمكانية التشغيل البيني الجغرافي المكاني عبر التنسيقات من خلال دعم تبادل البيانات بين أنظمة برمجيات جغرافية مكانية مختلفة من خلال دعم التنسيق الواسع الخاص به.
Opens and reads raster data from hundreds of geospatial file formats for analysis.
Anomalib is a PyTorch-based library for visual anomaly detection, offering a modular framework, a comprehensive model zoo, and a benchmarking suite designed for industrial defect detection. It provides a wide range of algorithms—including generative, discriminative, teacher-student, and vision-language approaches—that support unsupervised, few-shot, and zero-shot settings. The library enables deployment through model export to ONNX and OpenVINO for edge devices, and includes a no-code web application for training and inference. It also features a command-line interface for orchestrating multi
Anomalib loads image datasets from custom folders and standard formats, structuring them for model training and evaluation.
AliSQL is a fork of MySQL by Alibaba that extends the relational database management system with enhancements for high performance, scalability, and enterprise-grade availability. It retains the core MySQL identity as a SQL-based database for storing, organizing, and retrieving structured data, while adding optimizations for large-scale transactional and analytical workloads. The project differentiates itself through a set of Alibaba-specific improvements, including a columnar engine for accelerating analytical queries directly on MySQL tables, and a distributed, shared-nothing NDB Cluster en
MySQL imports structured and unstructured data into tables optimized for machine learning and generative AI pipelines.
bootstrap-fileinput is a Bootstrap-compatible HTML5 file upload widget and plugin. It provides a customizable interface for selecting and uploading multiple files, featuring integrated image previews, drag-and-drop support, and client-side validation for file types, sizes, and counts. The project includes a resumable file upload client that slices large files into chunks to ensure stability over intermittent connections and allow transfers to be paused and resumed. It also features a client-side image processor capable of resizing images and reading EXIF metadata to automatically correct imag
Aborts uploads or file loading by returning custom error messages from lifecycle events.
This project is a cloud infrastructure simulator and system design educational tool. It provides a sandbox environment for designing virtual network topologies and testing the resilience of system architectures under high traffic and simulated attacks. The platform functions as an infrastructure resilience sandbox where users can experiment with the architectural placement of load balancers, caches, and databases. It allows for the validation of auto-scaling strategies, rate limiting, and firewall configurations to prevent service degradation. The simulator covers network traffic management,
Implements memory caches and read replicas to optimize data retrieval and decrease database response times.
Nominatim is a self-hosted geospatial search engine and geocoding server that utilizes OpenStreetMap data. It provides a complete infrastructure for forward geocoding, converting addresses or place names into geographic coordinates, and reverse geocoding, translating coordinates into human-readable physical addresses. The project features a dedicated data importer that parses raw map data into a PostgreSQL geospatial database. It distinguishes itself through a configurable import pipeline that uses style files to filter map features and an importance-based ranking system to prioritize search
Loads map data into a database using style configurations to control the level of detail for boundaries and streets.
The Chromium Embedded Framework (CEF) is a framework for embedding a full Chromium web browser into native desktop applications, enabling them to render web content and execute JavaScript. It provides a multi-process browser runtime that manages isolated browser, renderer, and GPU processes to maintain application stability and security, along with a process sandboxing framework that restricts child process capabilities to prevent malicious web content from affecting the host system. CEF distinguishes itself through a comprehensive native JavaScript bridge library that bridges native applicat
Ships a resource loading and navigation interception system for customizing browser behavior from the host application.
TorchGeo is a PyTorch library designed for deep learning on geospatial data, providing a framework for building and training neural networks for tasks such as semantic segmentation, object detection, and change detection. It serves as a comprehensive pipeline for remote sensing, featuring specialized dataset loaders and multispectral image preprocessing tools. The library is distinguished by a dedicated remote sensing model zoo and extensive support for transfer learning, allowing users to integrate pre-trained weights optimized for specific satellite sensors. It also includes support for sel
Implements specialized loading for raster files incorporating custom coordinate reference systems and band selection.
PyTorchZeroToAll هو مورد تعليمي ومجموعة من البرامج التعليمية التي تركز على التعلم العميق وإطار عمل PyTorch. يوفر مساراً تعليمياً مهيكلاً لتنفيذ معماريات الشبكات العصبية، بدءاً من بناء جملة اللغة الأساسي والأساسيات وصولاً إلى تصميم النماذج المعقدة. يعمل المشروع كدليل تنفيذ لبناء أنواع مختلفة من الشبكات، بما في ذلك الشبكات الخطية، واللوجستية، والتلافيفية، والمتكررة. يغطي بشكل خاص سير عمل نمذجة التسلسل من خلال استخدام آليات الانتباه والشبكات على مستوى الأحرف. يغطي المورد أيضاً إعداد بيانات التعلم الآلي، بما في ذلك استخدام محملات مجموعات البيانات المخصصة، وأدوات معالجة النصوص، وترجمة ملفات المصدر الخام إلى tensors لخطوط أنابيب التدريب.
Demonstrates how to implement custom dataset loading classes to translate raw files into PyTorch tensors.