3 repos
Awesome GitHub RepositoriesData Processing Architectures
Design patterns and structural frameworks that define how data flows and is processed within a system.
Explore 3 awesome GitHub repositories matching data & databases · Data Processing Architectures. Refine with filters or upvote what's useful.
Awesome Data Processing Architectures GitHub Repositories
We'll search the best matching repositories with AI.
- BurntSushi/ripgrep
ripgrep is a command-line utility designed for searching through large file trees and source code repositories. It functions as a recursive text processor that traverses directories to locate and display matching patterns, serving as a high-performance alternative to traditional search tools. The tool distinguishes it
Rustclicommand-linecommand-line-tool - pathwaycom/pathway
Pathway is a high-performance data processing framework designed for building unified batch and streaming pipelines. It functions as an orchestrator for complex data transformations, utilizing a differential dataflow engine to process updates incrementally. By treating static datasets and continuous event streams with
Pythonbatch-processingdata-analyticsdata-pipelines - docling-project/docling
Docling is a modular framework designed for document parsing, layout analysis, and structured data extraction. It transforms unstructured files and web content into a unified, hierarchical data model that preserves the spatial and semantic relationships between text, tables, images, and layout elements. By normalizing
Pythonaiconvertdocument-parser
Explore sub-tags
- Buffered Stream ProcessorsSystems that utilize pre-allocated memory buffers to optimize sequential data throughput.
- Declarative Pipeline ConstructionDefining data workflows as static graphs optimized before execution.
- Exactly-Once Processing SemanticsGuarantees that each input record is processed exactly once despite system failures.
- Intermediate RepresentationsInternal data models that normalize diverse input formats into a consistent structure for uniform processing.