travisjeffery/jocko
Jocko
Jocko ist eine Cloud-native Event-Streaming-Plattform und ein verteilter Commit-Log, implementiert in Go. Es fungiert als verteilter Message-Broker, der Datenbeständigkeit und Hochverfügbarkeit durch die Replikation von Datensatzsequenzen über einen Cluster hinweg sicherstellt.
Das System ist als Zookeeper-freier Event-Streamer konzipiert und nutzt eine integrierte Konsens-Koordination, um den Cluster-Zustand und die Leader-Wahl ohne externe Koordinationsdienste zu verwalten. Es implementiert das Kafka-Wire-Protokoll, wodurch es mit bestehenden Ökosystem-Clients und Tools kommunizieren kann.
Die Plattform bietet Funktionen für das verteilte Log-Management, einschließlich der Verwendung eines Append-only-Commit-Logs für die Datenpersistenz. Sie enthält zudem automatisierte Mechanismen zur Datenaufbewahrung, die alte Datensätze basierend auf Zeitlimits oder Speicherplatzschwellenwerten bereinigen.
Das Projekt wird als Single-Binary-Laufzeitumgebung vertrieben.
Features
- Distributed Commit Logs - Functions as a distributed commit log ensuring fault-tolerant sequences of state changes.
- Kafka Compatible Message Brokers - Implements a distributed streaming platform compatible with the Kafka wire protocol for high-throughput messaging.
- Append-Only Message Logs - Implements an append-only log for high-throughput persistence and data durability.
- Cloud Native Message Queues - Provides a lightweight messaging cluster optimized for cloud-native deployment as a single binary.
- Distributed Consensus Stores - Maintains a consistent data state across multiple nodes using a consensus-based storage layer.
- Consensus Algorithms - Uses integrated consensus algorithms for cluster state management and leader election.
- Event Streaming Platforms - Operates as a high-throughput event streaming platform for real-time data ingestion.
- Zookeeperless Event Streamers - Provides a messaging system with built-in consensus coordination, eliminating the need for external services like Zookeeper.
- Cluster Coordination - Provides built-in cluster coordination to synchronize state without external dependencies like ZooKeeper.
- Distributed Messaging - Builds a scalable, fault-tolerant system for real-time message streaming across decentralized clusters.
- Replicated Log Management - Synchronizes sequential record logs across multiple nodes for high availability.
- Messaging Protocol Client Support - Provides communication bridges that enable existing Kafka clients to interact with the log system.
- Kafka Protocol Implementations - Implements the Kafka binary wire protocol to ensure compatibility with existing ecosystem tools.
- Data Retention & Cleanup - Automates the cleanup of legacy records to optimize storage and prevent disk exhaustion.
- Log Segment Purging - Automatically purges outdated data segments based on time limits or disk space thresholds.
- Single-Binary Distributions - Provides a single-binary distribution bundling the messaging engine and coordination logic.
- Distributed Systems - Provides a distributed system implementation in Go for simple, single-binary deployment.
- Messaging Systems - Kafka implementation in Go with cloud-native features.
