5 Repos
Techniques for fetching large datasets with minimal memory overhead through streaming and optimized mapping.
Distinct from High-Performance Data Infrastructures: Distinct from High-Performance Data Infrastructures: focuses on client-side memory efficiency during retrieval rather than server-side infrastructure
Explore 5 awesome GitHub repositories matching data & databases · Memory-Optimized Data Retrieval. Refine with filters or upvote what's useful.
Dapper is a high-performance micro-ORM and SQL object mapper for .NET. It functions as an ADO.NET extension library that adds data mapping capabilities directly to database connections, allowing SQL query results to be transformed into typed objects. The project prioritizes execution speed and low memory overhead by using intermediate language generation to map database columns to object properties. It further optimizes performance through the use of concurrent caching for mapping functions and literal value injection to improve database execution plans. The library covers a broad range of d
Fetches large datasets from a database with minimal memory overhead using row streaming and optimized mapping.
asyncpg is an asynchronous database driver and binary protocol client for PostgreSQL. It provides a non-blocking interface for executing SQL statements, streaming result sets, and managing data transfer between an application and a PostgreSQL database. The driver implements the PostgreSQL binary protocol directly to facilitate efficient data transfer and type conversion. It includes a connection pool to maintain and reuse open database connections, reducing the latency associated with repeated handshakes. The project covers a broad range of database integration capabilities, including atomic
Implements cursor-based result streaming to retrieve large datasets while minimizing application memory overhead.
This is a collection of classical algorithms and data structures implemented as a header-only C++ library. It provides a suite of tools for general algorithm implementation, including data structure management, graph theory analysis, and string processing. The library is distinguished by its specialized toolkits for cryptographic hashing and encoding, featuring implementations of MD5, SHA-1, and Base64. It also includes advanced capabilities for high-performance string processing via suffix trees and arrays, as well as computational number theory for primality testing and arbitrary-precision
Uses specialized filters like Bloom filters to optimize data membership lookups.
Zend Framework ist ein umfassendes Set entkoppelter Komponenten zum Aufbau modularer, ereignisgesteuerter Webanwendungen. Es implementiert eine MVC-Architektur zur Trennung von Geschäftslogik und Benutzeroberfläche und bietet ein strukturiertes Request-Handling-System durch eine sequentielle Middleware-Pipeline. Das Projekt zeichnet sich durch einen Factory-gesteuerten Dependency-Injection-Container aus, um die Objektinstanziierung zu automatisieren und Klassenlebenszyklen zu verwalten. Es enthält zudem eine umfassende Sicherheitssuite zur Verifizierung von Benutzeridentitäten und zur Einschränkung des Ressourcenzugriffs mittels Access Control Lists (ACL) und rollenbasierten Zugriffskontrolladaptern. Das Framework deckt ein breites Spektrum an Funktionen ab, einschließlich Datenbankabstraktion via Table- und Row-Gateways, Implementierungen von Remote Procedure Calls (RPC) für SOAP und JSON-RPC sowie ein Konsolen-Anwendungsframework für Befehlszeilenschnittstellen. Zusätzliche Funktionsbereiche umfassen Datenserialisierung, Eingabevalidierung, Sitzungsmanagement sowie Werkzeuge für E-Mail-Versand und Content-Internationalisierung.
Employs techniques for fetching large datasets with minimal memory overhead to prevent crashes in constrained environments.
This project is a static analysis tool and linter for Ruby on Rails designed to identify architectural smells and violations of best practices. It serves as a code quality linter, architectural auditor, security scanner, and performance analyzer for Rails applications. The tool evaluates the separation of concerns between controllers, models, and view templates to reduce technical debt. It identifies suboptimal coding patterns and enforces stylistic consistency, while specifically scanning for security vulnerabilities such as unprotected mass assignment in models. The analysis surface covers
Optimizes data retrieval for large datasets using streaming and batched finders to minimize memory overhead.