ydb-platform/ydb
Ydb
YDB este o bază de date SQL distribuită și un motor analitic conceput pentru scalabilitate orizontală și consistență puternică. Funcționează ca un sistem multi-model care suportă workload-uri tranzacționale și analitice printr-o arhitectură distribuită care oferă tranzacții ACID serializabile.
Sistemul se distinge prin compatibilitatea sa largă cu protocoalele, implementând protocolul PostgreSQL pentru driverele SQL standard și protocolul Kafka pentru mesagerie și streaming. Servește, de asemenea, ca o bază de date vectorială, suportând indecși vectoriali și căutări de tip approximate nearest neighbor pentru căutări semantice și embeddings.
Platforma gestionează datele folosind un model de stocare hibrid cu formate orientate pe rânduri și pe coloane, utilizând execuția interogărilor vectorizate pentru analize la scară de petabytes. Suprafața sa operațională include streaming de tip change data capture, cozi persistente de tip exactly-once și disponibilitate ridicată multi-zonă.
Deployment-ul și gestionarea ciclului de viață sunt susținute printr-un operator Kubernetes și provizionarea de tip infrastructure-as-code.
Features
- Analytical Data Engines - Processes petabyte-scale analytical queries using a vectorized query engine and columnar storage.
- Distributed SQL Databases - Provides a distributed SQL database architecture that combines horizontal scalability with strong transactional consistency.
- ACID Transaction Semantics - Guarantees serializable ACID transactions for strict data consistency across distributed nodes.
- Analytical Query Engines - Scales analytical workloads to petabytes using a vectorized query engine and distributed execution planner.
- Hybrid Storage Formats - Maintains data in both row and columnar formats to optimize for both transactional and analytical workloads.
- Cross-Zone Data Distribution - Distributes data across multiple availability zones to ensure access during simultaneous zone and rack failures.
- Data Recovery - Automatically restores data redundancy and service availability following disk, node, or datacenter failures.
- Declarative Query Languages - Provides a structured query language for declarative data retrieval and manipulation across a distributed cluster.
- Distributed Sharding Architectures - Implements distributed tablet sharding to enable horizontal scaling and balanced load distribution across nodes.
- Horizontal Scaling - Enables horizontal resource scaling by adding storage and computation nodes to the cluster.
- PostgreSQL-Compatible Databases - Implements the PostgreSQL wire protocol and SQL dialect for compatibility with standard database drivers.
- Real-Time Data Streaming - Integrates database tables with Kafka topics and processes unbounded data streams via change data capture.
- SQL Query Dialects - Provides a structured query language dialect for precise data management and retrieval.
- Vector Databases - Serves as a vector database supporting vector indexes and approximate nearest neighbor searches for embeddings.
- Vectorized Execution Engines - Utilizes a vectorized execution engine to process data in batches for high-performance analytical workloads.
- Column-Oriented Disk Storage - Organizes data independently by column to optimize analytical aggregates and maximize compression ratios.
- Distributed Data Workload Scaling - Distributes data and workloads across multiple nodes to ensure horizontal scalability and high availability.
- Hash-Based Data Distribution - Uses hash-based partitioning of keys to distribute data across shards and prevent insertion hotspots.
- Shard Rebalancing - Supports online addition or removal of nodes with automatic sharding to adapt to changing workloads.
- Backup & Recovery - Creates point-in-time database copies on object stores or filesystems for disaster recovery.
- Change Data Capture Streams - Provides real-time change data capture streaming of row-level modifications for external downstream processing.
- Erasure Coding - Uses erasure coding to provide high fault tolerance and data availability with lower storage overhead than mirroring.
- Tabular Row Storage - Stores data sequentially by row to optimize for transactional queries and fast single-record access.
- Database Wire Protocol Emulations - Implements various database wire protocols to allow standard drivers to interact with the internal system.
- Distributed Query Stream Processors - Runs streaming queries that automatically restart on failure and use checkpoints to persist state.
- Federated Data Query Engines - Executes federated queries to retrieve and process data from multiple external sources without migration.
- Indexing and Search - Supports a combination of primary, secondary, and vector indexes to optimize diverse data retrieval patterns.
- Multi-Tenant Resource Isolation - Runs multiple isolated databases within a single cluster using a shared storage layer to partition resources.
- PostgreSQL Protocol Compatibility - Supports the standard PostgreSQL wire protocol to ensure compatibility with existing PostgreSQL database drivers.
- Transactional Delivery Guarantees - Moves data between messaging topics and tables using transactional guarantees to ensure reliability.
- Vector Similarity Search - Executes approximate k-nearest neighbor queries on embeddings for semantic search and recommendations.
- Database Deployment Automation - Automates the installation and configuration of distributed database clusters across servers using orchestration tools.
- Operator-Managed Deployments - Employs a Kubernetes operator to automate the installation and lifecycle of storage and dynamic nodes.
- Database Cluster Deployments - Provides specialized orchestration for deploying and scaling distributed database clusters within container environments.
- Database Deployments - Offers automated deployment and management of database clusters in Kubernetes using a dedicated operator.
- Kafka Protocol Implementations - Implements the Kafka wire protocol to enable seamless integration with standard Kafka clients and topics.
- Database Protocol Implementations - Integrates with applications using the Kafka protocol for seamless messaging and streaming within the database.
- Persistent Queues - Provides persistent messaging queues with exactly-once delivery and automatic partitioning.
- Dynamic Shard Splitting - Automatically splits table partitions based on data size or request load to balance traffic across nodes.
