8 repositorios
Low-overhead processing of large-scale nested data collections using lazy operations.
Distinct from High Performance: Focuses on functional collection processing for memory efficiency, unlike candidates targeting hardware scalers or image processing.
Explore 8 awesome GitHub repositories matching data & databases · High-Performance Collection Processing. Refine with filters or upvote what's useful.
Hutool is a comprehensive suite of Java extensions designed to serve as a standard library extension. Its primary purpose is to reduce development boilerplate for common programming tasks and data manipulation through a collection of utility classes. The project provides specialized toolkits for database management using active record patterns and connection pooling, as well as network communication via a simplified HTTP client and asynchronous socket management. It includes security and identity capabilities such as symmetric and asymmetric encryption, image captcha generation, and JWT token
Implements Bloom filters for memory-efficient probabilistic membership verification.
Draft-js is a framework for building customizable rich text editors within React applications. It functions as a content editable framework that separates the underlying data model from the visual rendering layer, acting as a rich text content engine to manage complex text data and formatting. The project utilizes an immutable state management system to ensure consistent updates and predictable undo history. It manages editor state through persistent data structures, providing an immutable data state manager to prevent accidental mutation. The framework includes capabilities for high perform
Performs large-scale transformations and grouping on nested collections with minimal memory overhead.
EASTL is a C++ Standard Template Library implementation consisting of containers, iterators, and algorithms. It provides cross-platform data structures and a template-based algorithm library designed for use in resource-constrained game engine environments. The library focuses on game engine memory management, providing specialized utilities that ensure predictable memory allocation and high-performance access for real-time applications. These containers maintain consistent behavior across different operating systems and hardware platforms. The project covers high-performance C++ development
Provides high-performance containers for organizing and accessing data collections consistently across platforms.
RedisInsight is a graphical user interface and management tool for browsing, analyzing, and administering Redis databases. It provides a visual environment for exploring key-value data structures, managing database instances, and performing data analysis across different operating systems and deployments. The tool distinguishes itself by providing dedicated visual managers for complex operations, including a vector database manager for configuring embeddings and similarity searches, a query workbench for executing raw commands and Lua scripts, and a performance monitoring dashboard for tracki
Uses probabilistic algorithms to perform high-performance membership checks with significantly reduced memory overhead.
Pholcus is a distributed web crawling system designed for large-scale data scraping. It employs a master-worker distribution model to coordinate high-concurrency scraping tasks across a network of remote client nodes, enabling both horizontal and vertical data collection. The system features a hot-loadable rule engine that allows extraction and navigation logic to be updated at runtime without restarting the process. It handles dynamic content through headless browser integration and bypasses bot detection using proxy rotation, automated user authentication, and simulated human behavior. The
Uses static compiled code for high-performance scraping or dynamic files for hot-loading rules without restarting the system.
SpringSide 4 is an enterprise Java reference architecture and utility library built on the Spring Framework. It provides a pragmatic, best-practice application stack for building RESTful web services, web applications, and data access layers, along with a curated collection of high-performance utility classes for common operations like text, date, collection, reflection, concurrency, and I/O handling. The project distinguishes itself by combining a complete reference application scaffold with production-oriented infrastructure. It includes a JPA-based data access layer that automatically tran
Ships a curated collection of high-performance utility classes for common operations.
Esta biblioteca proporciona una colección de estructuras de datos especializadas para el lenguaje Swift que extienden la biblioteca estándar con tipos de contenedores avanzados. Incluye implementaciones para colas de doble extremo utilizando búferes circulares, colas de prioridad basadas en montículos min-max, y almacenamiento de bit-set y bit-array eficiente en memoria para valores booleanos. El proyecto cuenta con colecciones ordenadas que mantienen los elementos en orden clasificado mediante implementaciones de árboles B, así como conjuntos y diccionarios persistentes que utilizan árboles de prefijos comprimidos para compartir datos entre copias mutadas. También proporciona contenedores especializados que preservan el orden de inserción. La biblioteca cubre una gama de capacidades que incluyen la gestión de memoria de bajo nivel para búferes C y valores no copiables, almacenamiento de capacidad fija y el uso de hashing robin hood para optimizar la utilización de memoria y las velocidades de búsqueda.
Provides a comprehensive suite of high-performance Swift data structures, including priority queues and deques.
Esta es una biblioteca cliente de Ruby y driver de protocolo utilizado para integrar aplicaciones Ruby con bases de datos Redis. Funciona como una capa de comunicación que gestiona versiones de protocolo y proporciona una interfaz consistente para ejecutar operaciones de base de datos. La biblioteca soporta una variedad de topologías de despliegue, incluyendo instancias independientes, Redis Sentinel para descubrimiento de maestros de alta disponibilidad y conmutación por error, y Redis Cluster con enrutamiento de solicitudes consciente de slots y descubrimiento de nodos. También proporciona sharding del lado del cliente utilizando hashing consistente para distribuir datos a través de servidores independientes. Las capacidades amplias incluyen gestión de transacciones atómicas y scripting en Lua para transformaciones del lado del servidor, así como soporte para tipos de datos especializados como coordenadas geoespaciales y streams. El rendimiento se optimiza mediante el pipelining de comandos y extensiones de análisis nativas, mientras que la seguridad se maneja mediante cifrado SSL/TLS y autenticación mutua por certificados. El cliente incluye herramientas para orquestar topologías de bases de datos independientes y en clúster utilizando contenedores para pruebas de integración automatizadas.
Uses a high-performance extension driver to accelerate the processing of large data replies and high-volume pipelines.