uber/kraken
Kraken
Kraken is a distributed blob store and peer-to-peer Docker registry designed for the high-throughput distribution of container images. It functions as a decentralized content delivery network that shares image layers across a network of nodes to prevent bottlenecks at a central registry.
The system utilizes peer-to-peer blob distribution and distributed content addressing to maintain download speeds across large clusters. It implements asynchronous rule-based replication to synchronize image data between disparate geographical clusters.
The project covers pluggable external blob storage integration, layered image chunking, and identity-based upload verification to ensure data integrity and security.
Features
- Peer-to-Peer Mirrors - Functions as a peer-to-peer distribution system that shares image layers across nodes to reduce central server load.
- Distributed Blob Stores - Provides a distributed storage layer for large data blobs with asynchronous replication across geographical clusters.
- Content-Addressable Retrievals - Utilizes distributed content addressing to locate data fragments across the peer network using unique hashes.
- Container Image Registries - Operates as a distributed container registry that spreads image data across multiple nodes to prevent bottlenecks.
- Image Distribution Networks - Implements a high-throughput network for syncing and replicating container image blobs across multiple clusters.
- Image Replication - Synchronizes container image data across different geographical clusters using asynchronous replication.
- Cluster Image Distribution - Distributes large Docker image layers across a network of peers to maintain high download speeds.
- P2P - Functions as a decentralized P2P network to deliver container images to thousands of nodes simultaneously.
- P2P Container Registries - Provides a peer-to-peer distribution system for Docker images that shares data layers across nodes to accelerate deployments.
- Blob Storage Management - Integrates with external blob storage providers to manage the underlying data layer for container images.
- External Storage Integrations - Provides pluggable integration with external blob storage providers to decouple metadata from physical data.
- Identity-Based Verification - Ensures data integrity and security by validating the uploader identity before accepting new image layers.
- Rule-Based Replication - Implements asynchronous rule-based replication to synchronize image data between disparate geographical clusters.
- Layer Chunking - Breaks large container images into smaller verifiable pieces for more efficient distribution and partial transmission.
- Secure Registry Uploads - Verifies uploader identity and protects data integrity through encryption when pushing images to the distributed registry.
- Authenticated Uploads - Protects data integrity and secures image uploads using uploader identity verification and encryption.
- Container Registries - Scalable P2P registry for high-speed data distribution.
