14 Repos
Techniques for deferring the retrieval of related data until it is explicitly accessed.
Distinguishing note: Distinct from eager loading as it prioritizes initial query speed over immediate data availability.
Explore 14 awesome GitHub repositories matching data & databases · Lazy Loading Patterns. Refine with filters or upvote what's useful.
TypeORM is an object-relational mapper for TypeScript and JavaScript that bridges the gap between object-oriented application code and relational database tables. It provides a comprehensive data persistence layer that allows developers to define database entities using class decorators or configuration objects, enabling seamless interaction with data through object-oriented patterns. The project distinguishes itself through a flexible architecture that supports both the data mapper and repository patterns, alongside a fluent query builder that translates high-level method calls into platform
TypeORM loads related entity data only when explicitly accessed in code, reducing initial database query load and memory usage.
RedisDesktopManager is a NoSQL database manager and graphical user interface client for Redis. It serves as a desktop administrator for monitoring keys, managing memory, and executing commands on Redis servers. The application provides a visual environment for Redis data visualization, cache inspection, and database administration. It allows for the organization and editing of records across multiple data stores and server instances without the use of a command-line tool.
Retrieves subsets of keys or values from the server only as the user scrolls deeper into the list.
Ionicons is a library of hand-crafted scalable vector icons and an interface design asset pack used to build consistent user interfaces across web and mobile applications. It provides a collection of vector graphics designed for design system implementation and web component icon systems. The library includes a cross-platform icon set that automatically adapts its style based on the user's operating system to maintain a native look and feel. It also supports the integration of custom SVG assets through remote URL resolution to extend the available set of visuals. The system covers SVG asset
Streams SVG assets dynamically as they enter the viewport to improve page loading speeds.
Tushare is a financial data library for the Python programming environment that provides access to historical and real-time market information. It functions as a data interface for retrieving stock trading records, corporate financial statements, and macroeconomic indicators to support quantitative analysis and research. The library distinguishes itself by automatically transforming raw API responses into tabular data structures, allowing for direct integration with data analysis workflows. It manages access to these datasets through token-based authentication and utilizes schema-mapped parsi
Defers the retrieval of financial datasets until they are explicitly accessed to optimize memory usage.
This project is a comprehensive Python toolkit designed for natural language processing, research, and education. It functions as a linguistic data processor that provides a standardized framework for managing, cleaning, and analyzing large collections of annotated text corpora and lexical resources. The library distinguishes itself through its integration of both symbolic and statistical methods, allowing users to perform complex tasks ranging from rule-based grammar parsing to machine learning-driven classification. It offers a modular pipeline for text processing, enabling the transformati
Downloads and initializes linguistic models or corpora on demand to minimize memory footprint and optimize startup performance.
Peewee is a SQL object-relational mapper and query builder that provides an object-oriented interface for mapping application classes to relational database tables. It functions as a relational database toolkit for managing schemas, executing migrations, and handling complex table relationships. The project distinguishes itself by providing an asyncio database driver for non-blocking database operations, ensuring event loop responsiveness. It also supports semi-structured data storage, allowing the storage and querying of flexible JSON documents within traditional relational database systems.
Supports deferred loading of foreign key relationships to optimize initial query performance.
AllenNLP is a PyTorch-based research library and deep learning language toolkit designed for developing and training neural network architectures for linguistic tasks. It provides a distributed training system that coordinates data and gradients across multiple GPUs and a framework for integrating pretrained transformer architectures. The system distinguishes itself with a dedicated algorithmic bias mitigation tool used to identify and reduce bias in linguistic model predictions. It also includes model influence analysis to interpret predictions by calculating the influence of specific traini
Reads large datasets incrementally from disk to minimize memory usage during high-volume training.
WatermelonDB is an offline-first data synchronization engine and reactive database library designed for mobile and web applications. It provides a persistent storage layer backed by SQLite, enabling applications to maintain full functionality and data consistency while operating without an internet connection. The framework distinguishes itself through a reactive data binding system that automatically updates user interface components whenever underlying database records change. It utilizes schema-driven model mapping to generate type-safe interfaces and employs lazy object materialization to
Defers loading of database records until they are explicitly accessed to minimize memory usage.
Rerun is a multimodal data visualizer and robotics data logger designed for rendering synchronized streams of 3D spatial data, images, and time-series metrics. It functions as a tool for capturing high-frequency sensor data and AI outputs into a queryable columnar format, providing a dedicated interface for viewing MCAP recording files and analyzing physical environments. The project distinguishes itself as a machine learning dataset streamer, capable of feeding logged recordings directly into GPU buffers and PyTorch training pipelines without intermediate exports. It supports a high-performa
Streams recordings directly into machine learning frameworks via on-the-fly decompression and random seeking.
gs-quant is a quantitative finance library and financial data analytics toolkit. It serves as a framework for analyzing financial data, developing systematic trading strategies, and managing risk exposure for derivative products in global markets. The project provides tools for quantitative financial analysis, quantitative portfolio modeling, and the development of systematic trading strategies. It enables the calculation of risk for derivative products to structure and hedge positions across markets.
Implements lazy-loading patterns to defer the retrieval of large financial datasets until needed by analysis.
LanceDB is a vector database and columnar data store designed to function as a versioned dataset manager and vector search engine. It serves as a high-performance backend for indexing and retrieving high-dimensional embeddings, providing the foundation for machine learning data pipelines. The system distinguishes itself through a combination of cloud-native object storage and immutable version tracking, allowing for data time-travel and reproducible AI experiments. It integrates hybrid search capabilities, merging dense vector similarity with BM25 full-text search and SQL-like scalar filters
Loads data lazily from remote stores and projects specific columns to minimize network transfer.
Hibernate ORM ist ein Java-Object-Relational-Mapper und eine vollständige Implementierung der Jakarta Persistence API. Es dient als SQL-Datenbank-Abstraktionsschicht, die Java-Objektmodelle in relationale Datenbankschemata übersetzt, um Datenpersistenz und Lebenszyklen zu verwalten. Das Framework zeichnet sich durch ein Multi-Tenant-Datenisolations-Framework zur Trennung von Kundendaten innerhalb einer einzigen Datenbankinstanz aus. Es verfügt zudem über einen Datenbankschema-Generator, der automatisch relationale Strukturen basierend auf Entity-Mappings erstellt und aktualisiert. Das System deckt breite Funktionsbereiche ab, darunter Transaktionsmanagement, Concurrency-Locking-Kontrolle und temporale Datenverfolgung für Audit-Logging. Es bietet Tools zur Optimierung des Datenabrufs durch Entity-Graph-Strategien und unterstützt fortgeschrittene Datentypen wie Vektordaten und nationalisierte Zeichen. Das Projekt enthält eine umfassende Suite von Test-Utilities für die Persistenzschicht, einschließlich Datenbank-Dialekt-Filterung und Compliance-Tests für Spezifikationen.
Defers the retrieval of related data from the database until the object is explicitly accessed.
This project is a dataset management framework and cross-framework data loader that provides a unified interface for reading data formats compatible with TensorFlow, JAX, and PyTorch. It serves as a library of curated public datasets provided as data streams and includes tools for building, versioning, and documenting large-scale datasets. The system differentiates itself through a distributed data processing engine capable of managing massive datasets across clusters using parallelized pipelines. It utilizes builder-based construction to standardize how data is downloaded and prepared, while
Implements deferred loading of records as streams to handle datasets that exceed available system memory.
ParseUI-iOS ist eine Bibliothek wiederverwendbarer Interface-Komponenten für die Anzeige von Daten, die aus Remote-Backend-Diensten in mobilen Anwendungen abgerufen werden. Sie fungiert als Data-Binding-Framework, das Datenbankdatensätze direkt mit nativen Interface-Elementen verbindet und so den Boilerplate-Code reduziert, der für den Aufbau datengesteuerter Software erforderlich ist. Das Framework zeichnet sich durch vorgefertigte Widgets aus, die direkt in Backend-Datenmodelle integriert werden, wodurch Entwickler Informationen aus Cloud-Datenbanken automatisch rendern können. Durch die Nutzung von modellgesteuertem View-Binding beobachten diese Komponenten Änderungen in den zugrunde liegenden Daten und aktualisieren die Oberfläche entsprechend, was das Rapid Prototyping mobiler Anwendungen erleichtert. Die Bibliothek unterstützt die Standard-Interface-Konstruktion durch die Komposition nativer Plattform-Controls und stellt so ein konsistentes Erscheinungsbild über Geräte hinweg sicher. Sie verwaltet den Datenabruf durch Hintergrundoperationen und inkrementelle Lademuster, um die Reaktionsfähigkeit der Oberfläche zu erhalten und den Speicherverbrauch bei der Anzeige von Remote-Inhalten zu optimieren.
Implements incremental data loading during scrolling to optimize memory usage and improve initial performance.