14 repositorios
Techniques for deferring the retrieval of related data until it is explicitly accessed.
Distinguishing note: Distinct from eager loading as it prioritizes initial query speed over immediate data availability.
Explore 14 awesome GitHub repositories matching data & databases · Lazy Loading Patterns. Refine with filters or upvote what's useful.
TypeORM is an object-relational mapper for TypeScript and JavaScript that bridges the gap between object-oriented application code and relational database tables. It provides a comprehensive data persistence layer that allows developers to define database entities using class decorators or configuration objects, enabling seamless interaction with data through object-oriented patterns. The project distinguishes itself through a flexible architecture that supports both the data mapper and repository patterns, alongside a fluent query builder that translates high-level method calls into platform
TypeORM loads related entity data only when explicitly accessed in code, reducing initial database query load and memory usage.
RedisDesktopManager is a NoSQL database manager and graphical user interface client for Redis. It serves as a desktop administrator for monitoring keys, managing memory, and executing commands on Redis servers. The application provides a visual environment for Redis data visualization, cache inspection, and database administration. It allows for the organization and editing of records across multiple data stores and server instances without the use of a command-line tool.
Retrieves subsets of keys or values from the server only as the user scrolls deeper into the list.
Ionicons is a library of hand-crafted scalable vector icons and an interface design asset pack used to build consistent user interfaces across web and mobile applications. It provides a collection of vector graphics designed for design system implementation and web component icon systems. The library includes a cross-platform icon set that automatically adapts its style based on the user's operating system to maintain a native look and feel. It also supports the integration of custom SVG assets through remote URL resolution to extend the available set of visuals. The system covers SVG asset
Streams SVG assets dynamically as they enter the viewport to improve page loading speeds.
Tushare is a financial data library for the Python programming environment that provides access to historical and real-time market information. It functions as a data interface for retrieving stock trading records, corporate financial statements, and macroeconomic indicators to support quantitative analysis and research. The library distinguishes itself by automatically transforming raw API responses into tabular data structures, allowing for direct integration with data analysis workflows. It manages access to these datasets through token-based authentication and utilizes schema-mapped parsi
Defers the retrieval of financial datasets until they are explicitly accessed to optimize memory usage.
This project is a comprehensive Python toolkit designed for natural language processing, research, and education. It functions as a linguistic data processor that provides a standardized framework for managing, cleaning, and analyzing large collections of annotated text corpora and lexical resources. The library distinguishes itself through its integration of both symbolic and statistical methods, allowing users to perform complex tasks ranging from rule-based grammar parsing to machine learning-driven classification. It offers a modular pipeline for text processing, enabling the transformati
Downloads and initializes linguistic models or corpora on demand to minimize memory footprint and optimize startup performance.
Peewee is a SQL object-relational mapper and query builder that provides an object-oriented interface for mapping application classes to relational database tables. It functions as a relational database toolkit for managing schemas, executing migrations, and handling complex table relationships. The project distinguishes itself by providing an asyncio database driver for non-blocking database operations, ensuring event loop responsiveness. It also supports semi-structured data storage, allowing the storage and querying of flexible JSON documents within traditional relational database systems.
Supports deferred loading of foreign key relationships to optimize initial query performance.
AllenNLP is a PyTorch-based research library and deep learning language toolkit designed for developing and training neural network architectures for linguistic tasks. It provides a distributed training system that coordinates data and gradients across multiple GPUs and a framework for integrating pretrained transformer architectures. The system distinguishes itself with a dedicated algorithmic bias mitigation tool used to identify and reduce bias in linguistic model predictions. It also includes model influence analysis to interpret predictions by calculating the influence of specific traini
Reads large datasets incrementally from disk to minimize memory usage during high-volume training.
WatermelonDB is an offline-first data synchronization engine and reactive database library designed for mobile and web applications. It provides a persistent storage layer backed by SQLite, enabling applications to maintain full functionality and data consistency while operating without an internet connection. The framework distinguishes itself through a reactive data binding system that automatically updates user interface components whenever underlying database records change. It utilizes schema-driven model mapping to generate type-safe interfaces and employs lazy object materialization to
Defers loading of database records until they are explicitly accessed to minimize memory usage.
Rerun is a multimodal data visualizer and robotics data logger designed for rendering synchronized streams of 3D spatial data, images, and time-series metrics. It functions as a tool for capturing high-frequency sensor data and AI outputs into a queryable columnar format, providing a dedicated interface for viewing MCAP recording files and analyzing physical environments. The project distinguishes itself as a machine learning dataset streamer, capable of feeding logged recordings directly into GPU buffers and PyTorch training pipelines without intermediate exports. It supports a high-performa
Streams recordings directly into machine learning frameworks via on-the-fly decompression and random seeking.
gs-quant is a quantitative finance library and financial data analytics toolkit. It serves as a framework for analyzing financial data, developing systematic trading strategies, and managing risk exposure for derivative products in global markets. The project provides tools for quantitative financial analysis, quantitative portfolio modeling, and the development of systematic trading strategies. It enables the calculation of risk for derivative products to structure and hedge positions across markets.
Implements lazy-loading patterns to defer the retrieval of large financial datasets until needed by analysis.
LanceDB is a vector database and columnar data store designed to function as a versioned dataset manager and vector search engine. It serves as a high-performance backend for indexing and retrieving high-dimensional embeddings, providing the foundation for machine learning data pipelines. The system distinguishes itself through a combination of cloud-native object storage and immutable version tracking, allowing for data time-travel and reproducible AI experiments. It integrates hybrid search capabilities, merging dense vector similarity with BM25 full-text search and SQL-like scalar filters
Loads data lazily from remote stores and projects specific columns to minimize network transfer.
Hibernate ORM es un mapeador objeto-relacional para Java y una implementación completa de la Jakarta Persistence API. Funciona como una capa de abstracción de bases de datos SQL que traduce modelos de objetos Java a esquemas de bases de datos relacionales para gestionar la persistencia y el ciclo de vida de los datos. El framework se distingue por su arquitectura de aislamiento de datos multi-inquilino (multi-tenant) para separar la información de los clientes dentro de una misma instancia de base de datos. También cuenta con un generador de esquemas que produce y actualiza automáticamente estructuras relacionales basadas en mapeos de entidades. El sistema cubre áreas como la gestión de transacciones, control de concurrencia mediante bloqueos y seguimiento de datos temporales para auditoría. Proporciona herramientas para optimizar la obtención de datos mediante estrategias de grafos de entidades y admite tipos de datos avanzados como vectores y caracteres nacionalizados. El proyecto incluye un conjunto completo de utilidades de prueba para la capa de persistencia, incluyendo filtrado de dialectos de bases de datos y pruebas de cumplimiento de especificaciones.
Defers the retrieval of related data from the database until the object is explicitly accessed.
This project is a dataset management framework and cross-framework data loader that provides a unified interface for reading data formats compatible with TensorFlow, JAX, and PyTorch. It serves as a library of curated public datasets provided as data streams and includes tools for building, versioning, and documenting large-scale datasets. The system differentiates itself through a distributed data processing engine capable of managing massive datasets across clusters using parallelized pipelines. It utilizes builder-based construction to standardize how data is downloaded and prepared, while
Implements deferred loading of records as streams to handle datasets that exceed available system memory.
ParseUI-iOS es una biblioteca de componentes de interfaz reutilizables diseñados para mostrar datos obtenidos de servicios backend remotos dentro de aplicaciones móviles. Funciona como un framework de enlace de datos (data binding) que conecta registros de bases de datos directamente a elementos de interfaz nativos, reduciendo la cantidad de código boilerplate requerido para construir software basado en datos. El framework se distingue por proporcionar widgets preconstruidos que se integran directamente con modelos de datos de backend, permitiendo a los desarrolladores renderizar información desde bases de datos en la nube automáticamente. Al utilizar el enlace de vista basado en modelos, estos componentes observan cambios en los datos subyacentes y actualizan la interfaz en consecuencia, facilitando el prototipado rápido de aplicaciones móviles. La biblioteca admite la construcción de interfaces estándar mediante la composición de controles de plataforma nativos, asegurando una apariencia consistente en todos los dispositivos. Gestiona la recuperación de datos a través de operaciones en segundo plano y patrones de carga incremental para mantener la capacidad de respuesta de la interfaz y optimizar el uso de memoria durante la visualización de contenido remoto.
Implements incremental data loading during scrolling to optimize memory usage and improve initial performance.