28 repositorios
Interfaces for real-time access to incoming data streams.
Distinguishing note: Focuses on subscription-based access, distinct from batch query interfaces.
Explore 28 awesome GitHub repositories matching data & databases · Data Stream Subscriptions. Refine with filters or upvote what's useful.
graphql-engine is an automated GraphQL API engine that transforms database tables and relationships into a queryable GraphQL schema. It functions as a federation gateway and mapper, instantly generating APIs with built-in filtering, pagination, and mutations from existing databases and remote schemas. The project distinguishes itself through a fine-grained access control layer that enforces row-level and field-level permissions. It further provides a real-time data subscription server that converts standard queries into live streams and a system for triggering event-driven webhooks and notifi
Streams real-time updates to clients by tracking data changes and batching modifications using a cursor mechanism.
TDengine is a distributed time-series database designed for the high-speed ingestion, compression, and retrieval of timestamped metrics and sensor data. It functions as a SQL-compatible analytics engine, allowing users to perform complex operations on massive volumes of time-ordered information using standard relational syntax. The platform is built to serve as a backend foundation for industrial IoT environments, managing real-time data streams and device metadata through a cluster-based architecture. The system distinguishes itself through a distributed sharding architecture that uses consi
Provides built-in interfaces to subscribe to real-time data streams for downstream processing.
This project is a command-line utility and development framework designed to modify, extend, and customize the Spotify desktop client. It functions as a binary patching engine that injects custom scripts, stylesheets, and interface components directly into the host application, enabling users to alter visual themes and add new functionality. The tool distinguishes itself by providing a comprehensive development environment for building modular extensions and custom applications. It includes a hot-reloading pipeline for rapid iteration, a declarative library for constructing interactive UI pan
Listens for real-time updates from remote services to trigger callback functions within the client.
NATS Server is a high-performance, lightweight messaging system designed for cloud-native applications, edge computing, and distributed microservices. It functions as a distributed publish-subscribe broker that routes messages using hierarchical, dot-separated subject strings, enabling decoupled communication between services without requiring centralized broker lookups. The system supports core messaging patterns including asynchronous publish-subscribe, request-reply, and load-balanced queue processing. The platform distinguishes itself through a decentralized architecture that eliminates t
Routes messages using hierarchical subject matching to enable decoupled publish-subscribe communication.
gost is a multi-protocol proxy tunnel and secure tunneling server designed to route network traffic through encrypted connections. It functions as a traffic obfuscation gateway and a transparent proxy server capable of intercepting TCP and UDP traffic at the IP level. The project also includes a virtual network interface manager for creating TUN and TAP devices to intercept operating system packets. The system distinguishes itself through a chain-based request routing model, allowing traffic to pass through an ordered sequence of proxy nodes. It provides extensive transport-layer encapsulatio
Excludes dead or failing nodes from the selection pool based on failure counts.
Subconverter is a network utility designed to translate, merge, and filter proxy subscription configurations. It functions as a service that converts proxy links between various client-specific formats, ensuring compatibility across different applications and platforms. By providing a unified interface for managing diverse connection sources, the tool enables consistent network policy application and streamlined configuration management. The project distinguishes itself through an HTTP-based transformation interface that processes subscription data dynamically. It utilizes an in-memory pipeli
Refines proxy node lists by applying regex-based inclusion or exclusion rules to ensure only desired connections remain.
Apache Pulsar is a cloud-native distributed pub-sub messaging system designed for high-performance data ingestion. It functions as a geo-replicated data streamer and a multi-tenant event streaming platform, providing a serverless stream processing engine and a tiered storage messaging broker. The system distinguishes itself by separating serving layers from storage layers to allow independent scaling of compute and data retention. It features native geo-replication to synchronize messages across different geographical regions and employs a multi-layered tenant isolation model using authentica
Maintains individual consumer offsets using cursors to enable reliable message replay and independent stream tracking.
Cayley is a graph database engine designed for storing and querying interconnected data using a quad-based data model. It functions as an RDF quad store, managing information through subjects, predicates, objects, and labels. The system features a modular graph store architecture with pluggable backends, allowing it to swap between in-memory storage and various external persistent databases. It includes a GraphQL-inspired API and a dedicated data visualizer for the interactive exploration of nodes and edges. Query capabilities cover bidirectional path traversal and multi-syntax execution usi
Restricts the set of nodes based on predicate-object matches, range constraints, string matching, or identities.
Sub-Store is a proxy subscription management server that aggregates multiple subscription links into a single unified stream for distribution to various clients. It functions as a transformation pipeline that filters, modifies, and reformats proxy node metadata. The system acts as a cross-platform format converter to ensure compatibility across diverse client applications. It includes an encryption decryption gateway that uses private keys to handle age-standard encrypted subscription content and a cache-layered aggregator to reduce external requests. The server provides capabilities for dyn
Removes specific proxies from a list using regular expressions, regional markers, or custom scripts.
RisingWave is a cloud-native streaming database and real-time analytics engine that uses standard SQL to process continuous data streams. It functions as a streaming data lakehouse, combining the capabilities of a streaming SQL database with a platform that integrates streaming ingestion with open table formats. The system is distinguished by its use of the PostgreSQL wire protocol, allowing it to integrate with existing SQL tools and drivers. It employs a decoupled compute and storage architecture, persisting streaming state and materialized views in cloud object storage to enable independen
Delivers real-time updates from materialized views using a cursor-based subscription mechanism.
This is a Golang client library for interacting with a cloud native distributed messaging system. It provides the necessary tools for Go applications to exchange messages using publish-subscribe and request-reply patterns, as well as specialized clients for managing persistent streams and distributed storage. The library includes a JetStream client for durable message streaming and replay, a Key-Value store client for managing distributed state with versioning and watchers, and an Object Store client for the storage and retrieval of large binary files via chunked delivery. The implementation
Supports wildcard patterns in subject subscriptions to filter and match multiple data streams.
Clash-freenode is a tool that automatically fetches, aggregates, and generates subscription links for free proxy nodes from public sources, supporting Clash, V2Ray, and SSR formats. It operates by scraping Telegram channels and scanning public repositories to harvest proxy links, then converting them into static subscription URLs that clients can consume without manual updates. The project distinguishes itself by combining multiple aggregation methods—daily feed parsing, repository-based link harvesting, and Telegram channel scraping—into a single pipeline that produces protocol-compatible ou
Filters proxy nodes based on connectivity and latency checks to ensure only functional nodes are included.
re-frame es un framework funcional para construir aplicaciones de una sola página (SPA) en ClojureScript. Proporciona una base de datos centralizada e inmutable que sirve como única fuente de verdad para todo el estado de la aplicación, aplicando un flujo de datos unidireccional estricto donde los eventos activan transiciones de estado y actualizaciones de vista posteriores. El framework se distingue por un grafo de señales reactivas y un pipeline de middleware basado en interceptores. Al tratar la lógica de la aplicación como una secuencia de eventos basados en datos y efectos secundarios declarativos, desacopla la lógica de negocio de la capa de vista. Esta arquitectura permite a los desarrolladores gestionar transiciones de estado complejas y operaciones externas mediante funciones puras, asegurando que los efectos secundarios sean ejecutados por un intérprete separado en lugar de llamadas imperativas. El sistema incluye un conjunto completo de capacidades para gestionar la arquitectura de la aplicación, incluyendo la derivación de datos reactivos, la reconciliación de vistas basada en suscripciones y la gestión de estado impulsada por eventos. Admite flujos de trabajo de desarrollo avanzados como el rastreo de eventos, puntos de control de estado y la capacidad de crear stubs de efectos secundarios para pruebas aisladas. El proyecto está diseñado para integrarse con React, utilizando la reconciliación del DOM virtual para actualizar eficientemente las interfaces de usuario. Proporciona un conjunto robusto de utilidades para manejar preocupaciones transversales, gestionar grafos de flujo de datos complejos y coordinar operaciones asíncronas dentro de un pipeline de eventos secuencial y predecible.
Displays active subscription subjects for connected clients to facilitate debugging.
dlt es una herramienta de ingesta de datos en Python y framework de pipeline ETL diseñado para obtener datos de diversas fuentes y persistirlos en destinos estructurados. Funciona como un motor de inferencia de esquemas que detecta automáticamente tipos de datos y aplana estructuras JSON anidadas en tablas relacionales, moviendo datos desde fuentes a lakehouses, almacenes de datos o bases de datos vectoriales. El proyecto destaca por la generación de pipelines impulsada por IA, utilizando modelos de lenguaje de gran tamaño para crear código de extracción y conectores para APIs REST. También admite almacenamiento vectorial multimodal y población especializada de bases de datos vectoriales para soportar aplicaciones de IA y machine learning. El framework cubre una amplia gama de capacidades, incluyendo evolución automática de esquemas, carga incremental de datos mediante seguimiento de estado y validación de calidad de datos mediante la aplicación de contratos de datos. Proporciona herramientas para la normalización de datos relacionales, transformaciones pre y post-carga, y una variedad de adaptadores de destino para bases de datos SQL y almacenes de objetos en la nube. La observabilidad se maneja a través de paneles de ejecución de pipelines, seguimiento de linaje de columnas y verificación de versiones de esquema mediante hashes basados en contenido.
Uses cursors based on timestamps or IDs to load only data produced since the last execution.
Reactor Core es un kit de herramientas de programación reactiva y una base no bloqueante para componer pipelines de datos asíncronos en la JVM. Sirve como framework de procesamiento de flujos asíncronos y sistema de gestión de contrapresión (backpressure), permitiendo a los desarrolladores transformar, filtrar y combinar secuencias de eventos mientras regulan el flujo de datos entre productores y consumidores para evitar el agotamiento de recursos. La biblioteca se diferencia por un sofisticado sistema de planificación de concurrencia y control de flujo basado en la demanda. Desacopla el procesamiento de señales de hilos específicos utilizando un registro de planificadores y proporciona mecanismos para la propagación de metadatos inmutables conscientes del contexto a través de límites asíncronos. También cuenta con herramientas especializadas para la captura de trazas en tiempo de ensamblaje y planificación de tiempo virtual para facilitar la prueba de operadores basados en el tiempo. El proyecto cubre una amplia gama de capacidades, incluyendo procesamiento funcional de datos para agregación y ventanas de secuencias, una variedad de estrategias de recuperación de errores como reintentos con retroceso exponencial y utilidades para conectar API de callback heredadas o síncronas en flujos reactivos. Además, proporciona instrumentación para el monitoreo de pipelines y un conjunto de herramientas de prueba para verificar secuencias de señales.
Supports the creation of cold publishers that restart the data sequence for each unique subscriber.
Potpie is an LLM codebase analysis platform and multi-agent orchestration framework designed to act as an AI software engineer. It parses repositories into a structured code knowledge graph, enabling AI agents to perform multi-hop reasoning, dependency tracing, and grounded technical analysis across large codebases. The system distinguishes itself through a spec-driven development framework where agents generate detailed technical specifications and architecture plans before implementing multi-file code changes. It utilizes a durable execution engine to coordinate specialized AI personas for
Finds nodes in a knowledge graph by matching natural language queries against node descriptions.
RxPY es una librería de programación reactiva funcional y una librería de observables ReactiveX para Python. Funciona como un procesador de flujos asíncronos y un framework de coordinación basado en eventos, utilizado para construir pipelines de datos que reaccionan a cambios de estado o flujos de eventos a lo largo del tiempo. La librería proporciona un kit de herramientas para componer programas asíncronos y basados en eventos mediante secuencias observables y operadores. Se distingue por el uso de planificadores (schedulers) configurables para gestionar la concurrencia, el timing y los ciclos de vida de las suscripciones. El proyecto cubre una amplia gama de capacidades de procesamiento de flujos, incluyendo agregación, filtrado y combinación de datos. Proporciona mecanismos para la difusión de eventos, almacenamiento en búfer de secuencias y gestión de errores, así como herramientas para coordinar flujos observables con bucles de eventos asíncronos. Las pruebas y el aseguramiento de la calidad se apoyan en la simulación de tiempo virtual, el modelado con diagramas de mármol y la verificación de emisiones.
Provides interfaces for subscribers to attach handlers and react to real-time data stream emissions.
clashN is a cross-platform proxy manager and graphical user interface client for the Clash proxy core. It serves as a visual wrapper that enables the configuration of network tunnels and the routing of internet traffic to bypass network restrictions on desktop operating systems. The application functions by decoupling the management interface from the network engine, using a local API to dispatch configuration updates and control the proxy core. It includes a subscription converter that translates external provider profiles into standardized configuration schemas compatible with the underlyin
Deduplicates and filters proxy nodes across multiple subscriptions using regex and custom logic.
FastStream is an asyncio message broker framework for building event-driven applications in Python. It provides a unified interface and a multi-broker messaging abstraction layer that translates generic producer and consumer calls into broker-specific APIs. The framework features a built-in dependency injection container and uses decorators to route messages to asynchronous handler functions. It includes a documentation generator that extracts channel definitions and message formats from code to produce standardized AsyncAPI specifications. The project supports integration with Kafka, Rabbit
Subscribes to subjects using wildcard patterns to create flexible topic hierarchies.
Sqlbrite es un gestor de bases de datos SQLite reactivo y un wrapper ligero para Android. Funciona como una biblioteca SQL reactiva que convierte consultas estándar de SQLite en flujos observables, permitiendo que la aplicación emita nuevos resultados automáticamente cuando las tablas de la base de datos subyacentes cambian. El proyecto proporciona un mecanismo para rastrear cambios en tablas específicas de la base de datos, activando la actualización de consultas sin necesidad de sondeo manual. También incluye un gestor de transacciones atómicas para agrupar múltiples modificaciones SQL en unidades únicas, asegurando la integridad de los datos y evitando notificaciones redundantes. La biblioteca maneja la persistencia de datos locales en Android delegando las operaciones de esquema y la gestión del ciclo de vida de la base de datos al ayudante estándar de SQLite. Sus capacidades principales cubren la observación de datos en tiempo real y la gestión de transacciones de base de datos.
Transforms traditional SQLite result cursors into real-time data streams that automatically update the UI.