15 repositorios
Utilities for managing large binary blobs by automatically chunking data for storage.
Distinct from Object Storage Services: Distinct from Object Storage Services: focuses on the automatic chunking of binary data rather than general object storage management.
Explore 15 awesome GitHub repositories matching data & databases · Binary Object Chunkers. Refine with filters or upvote what's useful.
NATS Server is a high-performance, lightweight messaging system designed for cloud-native applications, edge computing, and distributed microservices. It functions as a distributed publish-subscribe broker that routes messages using hierarchical, dot-separated subject strings, enabling decoupled communication between services without requiring centralized broker lookups. The system supports core messaging patterns including asynchronous publish-subscribe, request-reply, and load-balanced queue processing. The platform distinguishes itself through a decentralized architecture that eliminates t
Manages large binary objects by automatically chunking data for efficient storage and retrieval.
FoundationDB is an ACID-compliant distributed transactional key-value store. It functions as a scalable database engine that ensures strict serializability and data consistency across a cluster of servers using a shared-nothing architecture. The system is distinguished by its multi-region replication capabilities, allowing data to be synchronized across different datacenters for high availability and disaster recovery. It utilizes optimistic concurrency control to manage distributed transactions and employs a majority-based coordination system to maintain cluster state. The platform provides
Automatically breaks large binary objects into smaller chunks for compatible storage within the database.
Dexie.js is a wrapper library for IndexedDB that provides a simplified interface for managing and querying structured data within the browser. It functions as a browser database manager used to maintain persistent application state and store binary blobs and records. The project serves as an offline-first data store that synchronizes browser data with remote servers to maintain consistency across sessions. It also acts as a reactive database store by monitoring data changes in real time to trigger automatic user interface updates, and functions as a client-side search engine for indexing and
Allows saving large binary objects and files directly alongside structured database records.
ABP is an opinionated architectural framework for building enterprise software solutions using .NET and ASP.NET Core. It serves as a structural toolkit for implementing domain-driven design and microservices patterns, providing a modular enterprise architecture where functionality is organized into independent, pluggable modules. The platform is specifically designed to support multi-tenant SaaS architectures, isolating data and configurations for multiple independent customers within a single application instance. It provides enterprise boilerplate infrastructure and pre-configured templates
Provides a standardized infrastructure layer for saving and retrieving large binary blobs and files.
Phabricator is a software development suite consisting of a collection of integrated web applications designed to manage the full software engineering lifecycle. It serves as a project management platform, issue tracking system, and code review tool. The suite provides capabilities for bug tracking and coordination, allowing teams to report and manage software defects and feature requests. It also facilitates peer code review workflows to manage proposed changes before they are merged into a repository. The platform includes tools for project task organization and general software developmen
Provides a storage system for managing large binary objects and attachments via the local filesystem.
LiteDB is a serverless, embedded NoSQL document database for .NET applications. It persists data into a single portable file, functioning as a BSON data store that resides within the application process rather than running as a separate server. The system is ACID compliant, utilizing write-ahead logging to ensure atomic, consistent, isolated, and durable transactions. It includes built-in encryption to provide secure local data storage and protect files on disk from unauthorized access. The project covers object-document mapping to convert classes into document formats, indexed search capabi
Provides specialized storage for large binary blobs to prevent document fragmentation.
LanceDB is a vector database and columnar data store designed to function as a versioned dataset manager and vector search engine. It serves as a high-performance backend for indexing and retrieving high-dimensional embeddings, providing the foundation for machine learning data pipelines. The system distinguishes itself through a combination of cloud-native object storage and immutable version tracking, allowing for data time-travel and reproducible AI experiments. It integrates hybrid search capabilities, merging dense vector similarity with BM25 full-text search and SQL-like scalar filters
Reads and writes large binary data using lazy-loading file objects to optimize memory usage.
Hazelcast is a distributed data platform that combines an in-memory data grid with a stream processing engine to support real-time analytics and event-driven applications. It functions as a partitioned, distributed key-value store that replicates data across cluster nodes to provide low-latency access and high availability. The platform also serves as a distributed SQL query engine, allowing users to execute standard SQL statements against both in-memory datasets and external data sources. What distinguishes Hazelcast is its use of a distributed consensus subsystem to maintain strongly consis
Enables storage of complex objects using configurable serialization methods.
Mongo-express es una interfaz administrativa basada en web para MongoDB, que proporciona una herramienta visual para gestionar bases de datos y colecciones sin el uso de una herramienta de línea de comandos. Construida como una aplicación Node.js, funciona como un editor de documentos y gestor de bases de datos para consultar, importar y exportar registros. El software incluye una interfaz administrativa compatible con proveedores de identidad OpenID Connect y OAuth2 para un acceso seguro de los usuarios. También cuenta con un monitor de rendimiento para ver estadísticas de salud de la base de datos y gestionar índices de colecciones para mejorar las velocidades de recuperación. El sistema cubre operaciones de datos amplias, incluyendo edición de documentos, consultas y la gestión de grandes activos binarios mediante GridFS. Proporciona controles administrativos para la visibilidad de la base de datos, configuración de conexiones cifradas y un modo de solo lectura para evitar la modificación accidental de datos.
Manages oversized binary data by utilizing the GridFS system to store files as chunks within the database.
attrs is a Python library that automatically generates initialization, representation, equality, hashing, and ordering methods from declarative class attribute definitions. At its core, it provides a class decorator metaprogramming framework that intercepts class creation to rewrite the class body, producing dunder methods without manual boilerplate. The library includes a comprehensive attribute validation toolkit with built-in validators for type checks, range constraints, regex matching, length limits, and logical composition of validation rules. The library distinguishes itself through it
Ships a binary serialization feature for attrs class instances using a compact binary format.
Rhai es un motor de scripting embebido y lenguaje de tipado dinámico diseñado para integrarse en aplicaciones Rust. Funciona como un compilador de árboles de sintaxis abstracta (AST) y una capa de interoperabilidad nativa, permitiendo a los desarrolladores mapear tipos y funciones de Rust en un entorno de scripting para comunicación bidireccional. El proyecto sirve como framework para crear lenguajes específicos de dominio (DSL) personalizables. Permite definir operadores, sintaxis y entornos de ejecución restringidos, facilitando la creación de lenguajes especializados con conjuntos funcionales a medida. El motor cubre una amplia gama de capacidades, incluyendo sandboxing con límites de recursos para una ejecución segura, organización modular del código y procesamiento integral de datos para tipos numéricos, de cadena y binarios. También proporciona herramientas para la manipulación de AST, serialización del estado de ejecución y observabilidad en tiempo de ejecución mediante la inspección del stack de llamadas y interfaces de depuración. Está diseñado para despliegue multiplataforma en cualquier CPU o sistema operativo compatible con el compilador nativo.
Handles packed arrays of bytes as large binary objects for efficient raw data storage.
H2 es un sistema de gestión de bases de datos relacionales compatible con JDBC, escrito en Java. Funciona como una base de datos SQL embebible que puede ejecutarse directamente dentro de un proceso de aplicación para eliminar la latencia de red, o como una base de datos en memoria para almacenamiento volátil de alto rendimiento. También incluye una consola basada en web para ejecutar comandos SQL y administrar esquemas. El sistema se caracteriza por sus modos de despliegue flexibles, incluyendo un modo servidor independiente para acceso remoto TCP/IP y un modo mixto para conectividad local y remota simultánea. Cuenta con una capa de emulación de dialectos y modos de compatibilidad que permiten imitar el comportamiento y la sintaxis de otros sistemas de bases de datos. El motor proporciona un amplio conjunto de capacidades que cubren transacciones ACID con control de concurrencia multiversión, soporte para datos geoespaciales y JSON, y funciones avanzadas de ventana analítica. Incluye herramientas para la preservación de datos mediante copias de seguridad comprimidas, restauración de scripts SQL y gestión de memoria fuera del heap (off-heap) para manejar grandes datasets. La base de datos se integra con aplicaciones utilizando controladores estándar de Java Database Connectivity y URLs de conexión.
Supports the storage and retrieval of large binary data objects using BLOB types.
uds es una utilidad de almacenamiento de binario a texto y gestor de almacenamiento de documentos en la nube. Convierte archivos binarios en fragmentos de texto codificados para facilitar el almacenamiento y la recuperación dentro de plataformas de documentos en la nube, específicamente para eludir las cuotas de almacenamiento en servicios como Google Drive. El sistema utiliza codificación base64 y división basada en fragmentos para dividir archivos binarios grandes en segmentos más pequeños que se ajustan a los límites de caracteres de documentos individuales en la nube. Reensambla estos archivos originales mediante la reconstrucción secuencial de bloques. El proyecto cubre la gestión de archivos en la nube, incluyendo la capacidad de listar, buscar y eliminar archivos almacenados. También proporciona capacidades de procesamiento por lotes para cargar y descargar múltiples archivos utilizando patrones y comodines, junto con utilidades para el formato de tamaño de archivo legible por humanos.
Saves binary data by splitting it into encoded documents to bypass cloud storage quotas.
fastjson2 is a high-performance Java library used for serializing and deserializing Java objects to and from JSON strings and byte arrays. It functions as a binary JSON serializer for encoding objects into optimized binary formats to reduce payload size and increase processing speed. The project includes a JSONPath query engine for extracting specific data and fields from JSON documents using standardized path expressions and filters. It also serves as a JSON schema validator, providing utilities to generate data contracts and validate JSON input against defined schemas to ensure structural c
Transforms Java objects into UTF-8 byte arrays for efficient caching and RPC communication.
Rusqlite is an embedded database interface and relational database driver that provides a client library for interacting with SQLite. It functions as an SQL query wrapper, enabling the management of local file-based or in-memory databases through a safe interface. The library allows for the extension of native database capabilities by implementing custom scalar functions, collations, and virtual tables. It also supports the embedding of the database engine directly into the application binary to remove external library dependencies. The project covers a broad range of capabilities including
Handles large binary data blobs through streaming I/O traits to minimize memory usage.