14 个仓库
Techniques for deferring the retrieval of related data until it is explicitly accessed.
Distinguishing note: Distinct from eager loading as it prioritizes initial query speed over immediate data availability.
Explore 14 awesome GitHub repositories matching data & databases · Lazy Loading Patterns. Refine with filters or upvote what's useful.
TypeORM is an object-relational mapper for TypeScript and JavaScript that bridges the gap between object-oriented application code and relational database tables. It provides a comprehensive data persistence layer that allows developers to define database entities using class decorators or configuration objects, enabling seamless interaction with data through object-oriented patterns. The project distinguishes itself through a flexible architecture that supports both the data mapper and repository patterns, alongside a fluent query builder that translates high-level method calls into platform
TypeORM loads related entity data only when explicitly accessed in code, reducing initial database query load and memory usage.
RedisDesktopManager is a NoSQL database manager and graphical user interface client for Redis. It serves as a desktop administrator for monitoring keys, managing memory, and executing commands on Redis servers. The application provides a visual environment for Redis data visualization, cache inspection, and database administration. It allows for the organization and editing of records across multiple data stores and server instances without the use of a command-line tool.
Retrieves subsets of keys or values from the server only as the user scrolls deeper into the list.
Ionicons is a library of hand-crafted scalable vector icons and an interface design asset pack used to build consistent user interfaces across web and mobile applications. It provides a collection of vector graphics designed for design system implementation and web component icon systems. The library includes a cross-platform icon set that automatically adapts its style based on the user's operating system to maintain a native look and feel. It also supports the integration of custom SVG assets through remote URL resolution to extend the available set of visuals. The system covers SVG asset
Streams SVG assets dynamically as they enter the viewport to improve page loading speeds.
Tushare is a financial data library for the Python programming environment that provides access to historical and real-time market information. It functions as a data interface for retrieving stock trading records, corporate financial statements, and macroeconomic indicators to support quantitative analysis and research. The library distinguishes itself by automatically transforming raw API responses into tabular data structures, allowing for direct integration with data analysis workflows. It manages access to these datasets through token-based authentication and utilizes schema-mapped parsi
Defers the retrieval of financial datasets until they are explicitly accessed to optimize memory usage.
This project is a comprehensive Python toolkit designed for natural language processing, research, and education. It functions as a linguistic data processor that provides a standardized framework for managing, cleaning, and analyzing large collections of annotated text corpora and lexical resources. The library distinguishes itself through its integration of both symbolic and statistical methods, allowing users to perform complex tasks ranging from rule-based grammar parsing to machine learning-driven classification. It offers a modular pipeline for text processing, enabling the transformati
Downloads and initializes linguistic models or corpora on demand to minimize memory footprint and optimize startup performance.
Peewee is a SQL object-relational mapper and query builder that provides an object-oriented interface for mapping application classes to relational database tables. It functions as a relational database toolkit for managing schemas, executing migrations, and handling complex table relationships. The project distinguishes itself by providing an asyncio database driver for non-blocking database operations, ensuring event loop responsiveness. It also supports semi-structured data storage, allowing the storage and querying of flexible JSON documents within traditional relational database systems.
Supports deferred loading of foreign key relationships to optimize initial query performance.
AllenNLP is a PyTorch-based research library and deep learning language toolkit designed for developing and training neural network architectures for linguistic tasks. It provides a distributed training system that coordinates data and gradients across multiple GPUs and a framework for integrating pretrained transformer architectures. The system distinguishes itself with a dedicated algorithmic bias mitigation tool used to identify and reduce bias in linguistic model predictions. It also includes model influence analysis to interpret predictions by calculating the influence of specific traini
Reads large datasets incrementally from disk to minimize memory usage during high-volume training.
WatermelonDB is an offline-first data synchronization engine and reactive database library designed for mobile and web applications. It provides a persistent storage layer backed by SQLite, enabling applications to maintain full functionality and data consistency while operating without an internet connection. The framework distinguishes itself through a reactive data binding system that automatically updates user interface components whenever underlying database records change. It utilizes schema-driven model mapping to generate type-safe interfaces and employs lazy object materialization to
Defers loading of database records until they are explicitly accessed to minimize memory usage.
Rerun is a multimodal data visualizer and robotics data logger designed for rendering synchronized streams of 3D spatial data, images, and time-series metrics. It functions as a tool for capturing high-frequency sensor data and AI outputs into a queryable columnar format, providing a dedicated interface for viewing MCAP recording files and analyzing physical environments. The project distinguishes itself as a machine learning dataset streamer, capable of feeding logged recordings directly into GPU buffers and PyTorch training pipelines without intermediate exports. It supports a high-performa
Streams recordings directly into machine learning frameworks via on-the-fly decompression and random seeking.
gs-quant is a quantitative finance library and financial data analytics toolkit. It serves as a framework for analyzing financial data, developing systematic trading strategies, and managing risk exposure for derivative products in global markets. The project provides tools for quantitative financial analysis, quantitative portfolio modeling, and the development of systematic trading strategies. It enables the calculation of risk for derivative products to structure and hedge positions across markets.
Implements lazy-loading patterns to defer the retrieval of large financial datasets until needed by analysis.
LanceDB is a vector database and columnar data store designed to function as a versioned dataset manager and vector search engine. It serves as a high-performance backend for indexing and retrieving high-dimensional embeddings, providing the foundation for machine learning data pipelines. The system distinguishes itself through a combination of cloud-native object storage and immutable version tracking, allowing for data time-travel and reproducible AI experiments. It integrates hybrid search capabilities, merging dense vector similarity with BM25 full-text search and SQL-like scalar filters
Loads data lazily from remote stores and projects specific columns to minimize network transfer.
Hibernate ORM 是一个 Java 对象关系映射器,也是 Jakarta Persistence API 的完整实现。它作为 SQL 数据库抽象层,将 Java 对象模型转换为关系数据库模式,以管理数据持久化和生命周期。 该框架以其多租户数据隔离框架而著称,可在单个数据库实例中分离客户数据。它还具有数据库模式生成器,可根据实体映射自动生成和更新关系结构。 该系统涵盖了广泛的功能领域,包括事务管理、并发锁定控制以及用于审计日志的时间数据跟踪。它通过实体图策略提供数据获取优化工具,并支持向量数据和国家化字符等高级数据类型。 该项目包含一套全面的持久层测试工具,包括数据库方言过滤和规范合规性测试。
Defers the retrieval of related data from the database until the object is explicitly accessed.
This project is a dataset management framework and cross-framework data loader that provides a unified interface for reading data formats compatible with TensorFlow, JAX, and PyTorch. It serves as a library of curated public datasets provided as data streams and includes tools for building, versioning, and documenting large-scale datasets. The system differentiates itself through a distributed data processing engine capable of managing massive datasets across clusters using parallelized pipelines. It utilizes builder-based construction to standardize how data is downloaded and prepared, while
Implements deferred loading of records as streams to handle datasets that exceed available system memory.
ParseUI-iOS 是一个可重用的界面组件库,旨在显示移动应用程序中从远程后端服务获取的数据。它作为一个数据绑定框架,将数据库记录直接连接到原生界面元素,减少了构建数据驱动软件所需的样板代码量。 该框架通过提供直接与后端数据模型集成的预构建小部件脱颖而出,使开发者能够自动渲染来自云数据库的信息。通过利用模型驱动的视图绑定,这些组件观察底层数据的变化并相应地更新界面,从而促进移动应用程序的快速原型设计。 该库通过组合原生平台控件支持标准界面构建,确保跨设备的一致外观。它通过后台操作和增量加载模式管理数据检索,以在显示远程内容时保持界面响应性并优化内存使用。
Implements incremental data loading during scrolling to optimize memory usage and improve initial performance.