apache/incubator-pulsar
Incubator Pulsar
Apache Pulsar is a cloud-native message queue and distributed publish-subscribe messaging system. It serves as a multi-tenant event streaming platform designed to route data streams for asynchronous communication between producers and consumers.
The system distinguishes itself through geo-replication, synchronizing data across multiple geographic regions to ensure high availability and low latency. It implements a multi-tenant architecture that provides isolation and resource management for millions of independent topics.
The platform covers high-throughput data streaming and event-driven data pipelines. Its capabilities include maintaining message ordering, managing multi-tenant access, and offloading cold data to tiered storage. System administration and resource provisioning are handled via a programmatic REST API.
Features
- Message Queues - Provides a cloud-native, scalable message queue that separates storage from serving for independent scaling.
- Publish-Subscribe Messaging - Implements a distributed publish-subscribe model for asynchronous communication between producers and consumers.
- Queue and Messaging - Provides reliable message queueing semantics to ensure data delivery across a distributed cluster.
- Event-Driven Data Pipelines - Provides reactive systems that route and buffer messages in a reliable sequence between distributed components.
- Regional Replication - Implements synchronization of message data across geographically distinct locations to ensure global availability and disaster recovery.
- High-Throughput Data Streaming - Optimizes the continuous flow and processing of massive volumes of event data across millions of topics.
- Event Stream Isolation - Provides built-in isolation and resource management for millions of independent topics across multiple tenants.
- Multi-Tenant Resource Isolation - Groups topics into namespaces with specific policies for quotas and security to provide strong multi-tenant resource isolation.
- Message Queues - Stores messages in a reliable buffer to manage event delivery between distributed components without data loss.
- Pub-Sub Messaging - Decouples message producers and consumers using topics and channels for asynchronous communication.
- Multi-Tenant Identity Management - Isolates user accounts and data within shared environments using authentication, authorization, and resource limits.
- Messaging Isolation Fabrics - Enforces multi-tenant data separation and controlled stream sharing within the messaging infrastructure.
- Message Data Offloading - Moves older messaging data from high-speed local disks to remote object storage to extend retention periods.
- Write Acknowledgement Strategies - Ensures data durability by requiring a majority of storage nodes to acknowledge a write before completion.
- Messaging Throughput Scaling - Dynamically adjusts resources to handle massive message volumes across millions of topics.
- Segmented Storage Architectures - Splits message logs into smaller segments distributed across a cluster for independent scaling of storage and serving.
- Message Ordering Guarantees - Ensures messages arrive at the consumer in the exact sequence they were sent to preserve data consistency.
- Stateless Serving Layers - Separates the stateless serving layer from the stateful storage layer to enable independent scaling of compute and disk.
- Data Pipelines - Distributed pub-sub messaging platform.
- Message Queues - Cloud-native distributed messaging and streaming platform.
Star history
How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
