go-mysql-org/go-mysql-elasticsearch

Features

• MySQL to Elasticsearch Synchronizers - Streams records from MySQL databases into Elasticsearch indices to enable real-time full-text search and analytics.
• Search Index Synchronizers - Maintains real-time consistency between MySQL databases and external Elasticsearch search engine indices.
• Elasticsearch Integrations - Moves data from MySQL to Elasticsearch in real time to enable fast full text search and complex querying.
• Change Data Capture Streams - Implements a change data capture stream that reads MySQL binary logs to trigger real-time search index updates.
• Change Data Capture Systems - Implements a change data capture system that streams database binary logs to maintain real-time search index synchronization.
• Database Replication Middleware - Acts as a synchronization layer that captures database state changes and broadcasts them to an external search index.
• Database Schema Mapping - Translates table and column names into specific index and field names using configurable mapping rules.
• Database-to-Index Mapping - Defines how specific tables or wildcard patterns map to target indices and data types.
• Real-Time Data Streaming - Streams updates from a database to a search engine to ensure the index reflects the current state of the data.
• Snapshot-Based Bootstrapping - Performs a full read of existing database tables to establish a baseline before switching to real-time streaming.
• Relational Search Indexing - Converts relational database tables into searchable documents while maintaining data types and custom field mappings.
• Relational-to-Document Mapping - Maintains one-to-many database joins by assigning parent identifiers to documents within the search index.
• Search Index Field Mapping - Translates database column names and types into search index fields using a predefined configuration map.
• Relational-to-Document Conversions - Transforms normalized MySQL database tables into hierarchical document-based formats for Elasticsearch indexing.
• Parent-Child Document Structures - Implements relational data denormalization by transforming database joins into parent-child document structures.
• Indexing Transformation Pipelines - Provides indexing transformation pipelines to cleanse and reshape database records before they are committed to the search index.
• Field Type Conversions - Changes data formats during synchronization, such as converting comma-separated strings into arrays for improved filtering.
• Document Processing Pipelines - Passes data through processing nodes to decode JSON or merge fields before the final indexing step.
• Field Transformations - Renames columns and converts data types to transform strings into arrays or integers into dates during synchronization.
• Ingest Pipeline Routing - Directs documents through pre-processing pipelines to perform tasks like decoding JSON or merging fields before indexing.
• One-to-Many Relationship Models - Maintains one-to-many joins by defining parent field properties that keep related documents organized in the index.
• Table Aggregation Patterns - Uses regular expression patterns to aggregate multiple partitioned database tables into a single search index.
• Regex Table Aggregation - Captures data from multiple tables matching a regex pattern and combines them into a single search index.
• Document Processing Pipelines - Provides sequential transformation stages to decode JSON or merge fields before indexing records.
• Index Document Linking - Links related documents in a search index using parent fields to support one-to-many data joins.

Star history