bup/bup
Bup
bup is a deduplicating backup manager and incremental backup system. It uses a Git packfile-based storage format to eliminate redundant data across files and versions, treating every incremental save as a full backup.
The system provides secure remote transport interfaces for transferring and managing backup data on remote servers via SSH. It also includes a backup repository browser available as both a web interface and a filesystem mount for exploring and retrieving files from snapshots.
The project covers broad capability areas including disaster recovery, repository administration, and storage optimization. It utilizes rolling checksum chunking for data reduction, garbage collection for space reclamation, and integrity auditing to validate backup archives. Recovery features support target version restoration, sparse file handling, and the restoration of POSIX1e ACLs.
Features
- Storage Deduplication - Utilizes a Git packfile-based storage format to eliminate redundant data across files and versions.
- Deduplication - Eliminates redundant data across files and versions using a packfile format to minimize disk space usage.
- Backup Repositories - Manages structured storage formats for long-term data archival and repository health.
- Content-Addressable Storage - Implements a content-addressable storage architecture to ensure data uniqueness and verify backup integrity via cryptographic hashes.
- Disaster Recovery Workflows - Implements automated processes for off-site data persistence and recovery from catastrophic failures.
- Delta-Compressed Packfiles - Utilizes a compressed packfile format with delta compression to enable global deduplication and efficient incremental saves.
- Packfile Backup Systems - Uses the Git packfile format for global deduplication and fast incremental saves.
- Reference-Based State Tracking - Tracks backup versions using a system of commits and tags to allow rapid recovery of any previous state.
- Incremental Backup Storage - Saves only changes between file versions to reduce storage while treating every save as a full backup.
- Remote Backup Protocols - Manages remote data backup repositories over network protocols using secure shell connections.
- SSH Data Transfers - Provides secure streaming of backup data to remote servers using the SSH protocol.
- Merkle-Tree Storage Engines - Organizes file hierarchies into Merkle trees of hashes to represent snapshots and efficiently track changes over time.
- Backup Utilities - Provides a system for automating file backups and recovery using incremental saves.
- Deduplication Backup Tools - Optimizes storage by identifying and storing unique data chunks to eliminate redundant data.
- Remote Data Synchronization - Transfers and synchronizes backup data to remote destination servers via secure shell connections.
- Archive Integrity Auditing - Provides a process that scans backup archives to identify corruption and verify the consistency of stored data.
- Snapshot Tagging - Allows assigning human-readable labels to specific backup points using persistent reference updates.
- Snapshot Mounting - Exposes specific snapshots as mountable filesystems to retrieve files using standard system tools.
- Backup Content Listing - Implements an optimized process for displaying files and directories within large backup data sets.
- Data Recovery - Reconstructs backup data lost to disk corruption using parity information and redundancy.
- Versioned Data Restorations - Retrieves a specific previous version of backed-up data and restores it to a target directory.
- Data Restoration Tools - Provides utilities to copy data from a backup repository back to the filesystem for recovering previous file versions.
- Filesystem Mounts - Exposes backup repositories as a virtual filesystem to allow direct access and browsing via the operating system.
- Repository Link Validation - Scans the repository to detect and report broken links between trees and commits.
- Index Health Verification - Provides a utility to check backup indices for missing components to ensure data recoverability.
- Rolling Checksum Chunking - Employs rolling checksum chunking to minimize the amount of new data stored during backups.
- Storage Garbage Collection - Provides a garbage collection process to identify and remove unreferenced data objects to reclaim disk space.
- Store Consistency Validation - Checks for missing objects and verifies internal references within the store to ensure total data consistency.
- Save Referential Integrity - Performs referential integrity checks on individual saves to ensure they are complete without requiring a full restore.
- Backup Archive Browsers - Provides a web interface and filesystem mount to browse backup archives as local filesystems.
- Storage Reclamation Utilities - Deletes obsolete branches and saves to remove unreferenced data and recover physical disk space.
- File Restoration - Extracts files from a chosen backup destination or commit to recover data to a previous state.
- Remote Data Restorations - Retrieves and restores files directly from a remote repository to a local destination.
- Packfile Optimization - Optimizes the backup store by rewriting packfiles and removing redundant data to improve efficiency.
- Remote Data Streaming - Streams backup data to remote servers over encrypted SSH shells for secure offsite storage without local staging.
- FUSE Virtual Filesystems - Exposes backup snapshots as a virtual filesystem using FUSE, allowing data browsing via standard operating system tools.
- Policy-Based Retention - Implements a policy-based utility to prune aged saves while maintaining key historical recovery points.
- Backup and Recovery - Efficient incremental backup system using git packfile formats.
- Backup Solutions - Incremental backups with rolling checksums and FUSE filesystem support.
- Desktop Productivity Tools - Efficient backup system using git-based incremental saves.
