5 repositorios
Formats for organizing aggregated data blobs with headers for efficient streaming.
Distinguishing note: Focuses on the internal structure of pack files for streaming and indexing.
Explore 5 awesome GitHub repositories matching data & databases · Pack File Structures. Refine with filters or upvote what's useful.
This project is a command-line utility designed for secure, content-addressable data archiving. It functions as an encrypted backup tool that stores data as deduplicated chunks, ensuring that every piece of information is identified by a cryptographic hash to maintain integrity across all backups. By applying strong encryption and message authentication codes to both data and metadata, the software prevents unauthorized access and detects potential tampering. The tool distinguishes itself through a backend-agnostic storage abstraction that allows users to maintain repositories across diverse
Organizes backup data into pack files containing multiple blobs with an encrypted header for efficient indexing.
This project is a deep learning framework designed for constructing, training, and deploying neural networks across diverse hardware environments. It functions as a high-performance tensor computation library that provides both imperative and symbolic programming interfaces, allowing developers to balance flexible, step-by-step model building with the efficiency of compiled computation graphs. The framework distinguishes itself through a hybrid execution engine that integrates declarative graph compilation with imperative runtime logic. It supports scalable, distributed training across multip
Organizes aggregated data blobs into splittable binary formats for efficient streaming during training.
ZeroByte is a backup management platform built around the Restic backup engine, providing encrypted, deduplicated, and compressed snapshots across multiple storage backends. It offers a web interface for scheduling, monitoring, and managing backup operations, with support for cron-based job scheduling and configurable retention policies that automatically prune older snapshots. The platform distinguishes itself through comprehensive multi-protocol volume mounting, allowing backup ingestion from NFS, SMB, WebDAV, SFTP, and rclone-backed sources alongside local directories. It includes a snapsh
Tunes pack file size to optimize performance for different storage backends and repository scales.
HexFiend is a native macOS hex editor designed for opening and editing binary files of any size without loading them entirely into memory. It uses a 64-bit addressable data model with memory-mapped file access, enabling navigation of multi-gigabyte files while maintaining byte-level undo for insert and delete operations. The editor distinguishes itself through a diff algorithm for binary file comparison that accounts for insertions and deletions, and a template-based structure parsing system that uses user-defined scripts to visualize binary file layouts. It also includes a typed data interpr
Uses scripts to parse and display the structure of binary files as structured views.
UEFITool es una herramienta de análisis de firmware y editor de imágenes diseñado para analizar, visualizar y modificar imágenes de firmware UEFI. Funciona como un analizador de especificaciones que convierte datos binarios de firmware en formatos estructurados basados en las especificaciones estándar de interfaces de plataforma. La herramienta proporciona una interfaz visual para analizar la estructura de árbol y los elementos de datos internos de las imágenes de firmware de plataforma. Permite la modificación de imágenes de firmware para ajustar los entornos de arranque del sistema e incluye capacidades para buscar patrones de datos específicos dentro de imágenes binarias. El software cubre una gama de capacidades de gestión de firmware, incluida la extracción de componentes de firmware al sistema de archivos para análisis externo y la visualización de datos de firmware sin procesar para verificar la integridad interna.
Parses and displays the internal binary structure of UEFI firmware images.