6 Repos
Connecting to and registering streaming data sources like Kafka for real-time feature ingestion.
Distinguishing note: No candidate covers registering a Kafka stream as a data source for a feature store; candidates are about authentication or encryption services.
Explore 6 awesome GitHub repositories matching data & databases · Streaming Data Sources. Refine with filters or upvote what's useful.
This repository is a collection of practical code snippets and implementation patterns for Flutter and Dart. It serves as a comprehensive guide and reference for asynchronous programming, state management patterns, and UI component design. The project provides advanced language reference material covering generics, reflection, factory constructors, and null-aware operators. It also includes specific utilities for manipulating Dart collections, such as helper methods for transforming and filtering maps, lists, and iterables. The coverage extends to high-level capabilities including asynchrono
Renders live data sources in the interface with automatic updates as new events arrive.
Feast is an open-source feature store for machine learning that provides a central platform for defining, storing, and serving features across both training and inference workflows. It operates as a declarative system where feature definitions are written as code in Python files, synchronized to a central registry, and made available for low-latency online retrieval or point-in-time correct historical joins for training datasets. The project abstracts storage behind a pluggable architecture, allowing offline and online backends to be swapped without changing retrieval logic, and coordinates ma
Registers Kafka streams as data sources for real-time feature computation and ingestion.
Ibis is a portable Python dataframe library and multi-backend query engine that provides a unified interface for executing data transformations across diverse compute engines. It functions as a Python SQL expression compiler and dialect transpiler, allowing users to define data logic once and execute it across cloud warehouses, embedded databases, and distributed clusters without rewriting code. The project distinguishes itself through a database backend abstraction that decouples transformation logic from the underlying execution engine. It enables polyglot data workflows by mixing raw SQL s
Allows registering and managing streaming data sources, such as Kafka, for real-time ingestion.
Materialize is a streaming SQL database that continuously ingests live data from sources such as Kafka, Redpanda, PostgreSQL, and MySQL, and incrementally maintains materialized views. It provides a PostgreSQL-compatible query engine that accepts standard SQL over the PostgreSQL wire protocol, enabling any existing SQL client or BI tool to query real-time data. The system also includes a Model Context Protocol (MCP) server that exposes live materialized view data to AI agents, providing fresh context without polling. Materialize distinguishes itself through its ability to offer configurable c
Reads live data from PostgreSQL, MySQL, Kafka, and webhooks for continuous streaming ingestion.
Nuclio is a high-performance serverless framework designed for Kubernetes that automatically executes user functions when events arrive from HTTP endpoints, message queues, or streaming data platforms. It processes hundreds of thousands of events per second per function instance through efficient parallel workers, and can allocate functions to run on either CPU or GPU hardware to match workload requirements for data processing or machine learning tasks. The platform scales function instances down to zero when idle and wakes them on demand based on incoming event load, while providing an event
Connects functions to streaming data sources so they run automatically when new events arrive.
RavenDB is a multi-model NoSQL document database designed for high-performance, ACID-compliant data storage. It persists structured information as schema-flexible JSON documents and utilizes a unit-of-work session pattern to track entity changes and batch modifications into atomic transactions. The platform is built on a distributed architecture that supports horizontal scaling through sharding and ensures high availability via multi-node, master-to-master cluster replication. The database distinguishes itself through a self-optimizing query engine that automatically creates and maintains ind
Supports real-time streaming ingestion from external message brokers like Kafka or RabbitMQ.