257 repository-uri
Platforms for processing and delivering data streams in real-time.
Distinguishing note: Focuses on event-driven streaming integrated into the database layer.
Explore 257 awesome GitHub repositories matching data & databases · Real-Time Data Streaming. Refine with filters or upvote what's useful.
Acest proiect este un director curatoriat de comunitate cu endpoint-uri de servicii REST și GraphQL, conceput pentru a ajuta dezvoltatorii să descopere și să integreze surse de date terțe. Funcționează ca un registru centralizat unde serviciile externe sunt organizate pe domenii pentru a facilita prototiparea rapidă a software-ului și dezvoltarea aplicațiilor. Registrul se bazează pe un model de contribuție peer-reviewed, utilizând controlul distribuit al versiunilor pentru a gestiona actualizările și a asigura acuratețea endpoint-urilor listate. Pentru a menține o calitate ridicată a datelor, proiectul folosește validarea bazată pe schemă pentru toate trimiterile primite și compilează datele structurate într-un site web static, ușor de căutat, pentru o regăsire eficientă. Directorul acoperă un spectru larg de capabilități de integrare, inclusiv regăsirea datelor financiare, servicii de geolocalizare și diverse API-uri utilitare pentru sarcini precum detectarea limbajului, procesarea media și verificarea identității. Prin furnizarea unui index centralizat al acestor servicii, proiectul sprijină dezvoltatorii în identificarea furnizorilor de date fiabili pentru diverse cerințe funcționale.
Provides structured interfaces for retrieving global weather information.
Acest proiect este un director curatoriat de comunitate cu software open-source conceput pentru implementarea în medii de server private și laboratoare de acasă (home labs). Servește drept resursă cuprinzătoare pentru descoperirea alternativelor independente, auto-găzduite, la serviciile cloud mainstream, permițând utilizatorilor să mențină proprietatea deplină a datelor și controlul asupra infrastructurii lor digitale. Directorul este structurat printr-o taxonomie ierarhică ce organizează o colecție vastă de aplicații în categorii logice, variind de la gestionarea media și analiza datelor la comunicare privată și instrumente de productivitate în echipă. Se distinge printr-un proces colaborativ de peer-review, unde membrii comunității validează calitatea și relevanța fiecărei trimiteri pentru a se asigura că directorul rămâne precis și fiabil. Proiectul acoperă o suprafață largă de capabilități, inclusiv automatizarea infrastructurii, implementarea serviciilor bazate pe containere și gestionarea configurației declarative. Aceste instrumente ajută utilizatorii să mențină medii de server reproductibile și să gestioneze dependențele complexe ale serviciilor pe hardware privat. Directorul este menținut ca un repository controlat prin versiuni, asigurându-se că toate actualizările și modificările conduse de comunitate sunt urmărite și transparente.
Routes and publishes live video or audio feeds across multiple protocols to enable low-latency distribution and recording.
OpenCV is an open-source computer vision library and visual analysis toolkit. It provides a framework for processing static images and dynamic video frames to analyze visual data and extract information using deep learning. The project functions as a real-time image processing framework, enabling the execution of vision algorithms on live video streams for immediate analysis and data processing. The toolkit covers a broad range of capabilities including image pattern recognition, real-time video analysis, and visual data extraction. It also supports automated visual inspection for detecting
Executes vision algorithms on live video streams for immediate visual analysis and data processing.
Dubbo is a Java RPC framework and microservices governance platform designed for high-performance remote procedure calls in distributed architectures. It provides the foundational components necessary to connect distributed services across a network, including a binary data serialization library and a distributed service registry. The platform distinguishes itself through a comprehensive governance suite that manages service discovery, load balancing, and traffic routing. It enables precise control over network traffic via conditional routing and a pluggable extension mechanism based on a ser
Provides high-performance data streaming capabilities between clients and servers using the gRPC protocol.
This repository serves as a comprehensive research platform and toolkit for advancing machine learning, quantum computing, and large-scale scientific data analysis. It provides foundational frameworks for developing complex algorithmic systems, offering the necessary infrastructure for distributed training, computational graph execution, and high-performance model development. The project distinguishes itself by integrating specialized research domains with robust, privacy-preserving methodologies. It supports diverse scientific discovery through tools for quantum simulation, physics-informed
Combines diverse meteorological inputs from satellite and gauge-based sources into a unified framework to improve predictive accuracy.
SurrealDB is a multi-model database engine designed to store and query document, graph, relational, and vector data within a single ACID-compliant platform. It functions as an AI-native data store, integrating vector search, graph traversal, and machine learning model execution directly into its query layer. By providing a unified declarative query language, the platform eliminates the need for external middleware to synchronize data across different storage models. The platform distinguishes itself through its ability to manage agent memory and complex workflows natively. It allows developer
Powers reactive, real-time experiences using built-in subscriptions, event triggers, and streaming updates.
graphql-engine is an automated GraphQL API engine that transforms database tables and relationships into a queryable GraphQL schema. It functions as a federation gateway and mapper, instantly generating APIs with built-in filtering, pagination, and mutations from existing databases and remote schemas. The project distinguishes itself through a fine-grained access control layer that enforces row-level and field-level permissions. It further provides a real-time data subscription server that converts standard queries into live streams and a system for triggering event-driven webhooks and notifi
Converts standard GraphQL queries into live queries that stream real-time updates to clients.
SRS is a real-time media server designed to ingest, route, and distribute live audio and video streams across various transport protocols. It functions as a multi-protocol stream relay, including a dedicated RTMP media gateway and a WebRTC signaling server to coordinate peer-to-peer media exchanges. The system features a multi-protocol relay engine that transforms incoming media packets between different transport formats without re-encoding. This allows it to serve as a video delivery proxy that routes live media from a single source to multiple concurrent viewers using diverse delivery prot
Provides a multi-protocol relay engine to synchronize and forward live media streams between diverse transport protocols.
Hyperframes is an HTML-to-video rendering engine and composition tool that transforms web layouts and CSS into encoded video files. It functions as a headless browser video pipeline and a distributed video rendering framework, allowing users to create seekable animations and programmatic motion designs using HTML, CSS, and JavaScript. The project differentiates itself as an AI agent video orchestrator, enabling the automation of video scripts and compositions through natural language prompts. It supports distributed video encoding by splitting rendering tasks across multiple serverless functi
Pipes captured browser frames directly to the encoder in real time to avoid writing intermediate files to disk.
Seata is a distributed transaction coordinator designed to ensure data consistency and atomicity across microservices. It provides a centralized framework for managing global transactions, preventing partial data updates across different databases and services. The project implements multiple transaction modes to balance consistency and performance. This includes an automatic mode that uses rollback logs to coordinate compensation without modifying business logic, a try-confirm-cancel pattern for resources lacking native ACID support, and a saga orchestration engine for managing long-lived bu
Uses gRPC for high-performance bidirectional data streaming between clients and coordinators.
Kotaemon is an orchestration framework designed for building modular, agentic workflows that integrate document processing, retrieval-augmented generation, and multi-step reasoning. It provides a comprehensive platform for developing document-based question answering systems, allowing users to chain language models, prompt templates, and external tools into complex, automated pipelines. The system distinguishes itself through a highly modular architecture that emphasizes component-based composition and schema-driven data exchange. It supports autonomous agents capable of decomposing complex q
Streams incremental results from reasoning pipelines to the user interface in real-time.
RxSwift is a reactive programming library for Swift that provides a framework for managing push-based data flows and composing asynchronous, event-based programs. It utilizes observable sequences and functional operators to transform and filter asynchronous sequences through a declarative approach. The library is distinguished by its ability to link asynchronous data streams directly to user interface elements, automating view updates via reactive data binding. It includes specialized tools for tracking UI control properties and events on the main thread, as well as the ability to encapsulate
Distributes a single subscription across multiple observers with support for replaying historical elements.
This project is a reactive, offline-first NoSQL database engine designed for JavaScript applications. It provides a robust framework for managing application state by synchronizing data across browsers, mobile devices, and server-side runtimes. By treating local storage as the primary source of truth, it enables applications to remain functional without network connectivity, automatically reconciling changes with remote backends once a connection is restored. The database distinguishes itself through a modular architecture that supports cross-environment synchronization and high-performance d
Provides observable queries and documents that push state changes to the application for dynamic, responsive user interfaces.
This project serves as a comprehensive technical reference for the architecture and design of data-intensive applications. It provides a structured analysis of the fundamental principles required to build reliable, scalable, and maintainable software systems, covering the core trade-offs inherent in modern data infrastructure. The repository explores the mechanics of distributed data management, including strategies for replication, partitioning, and achieving consensus across multiple nodes. It details the design of storage engines, indexing techniques, and transaction management models, whi
Matches incoming documents against stored queries in real time to notify users when data meets specific criteria.
RocketMQ is a distributed messaging and streaming platform designed for building event-driven applications. It serves as middleware to decouple services using publish-subscribe and request-reply patterns, and functions as a transactional messaging system that ensures atomicity by linking message delivery to local transaction outcomes. The platform includes specialized capabilities as a Kubernetes-native message broker for container orchestration environments and an MQTT broker for ingesting event data from mobile applications and hardware terminals. The system covers high-throughput data str
Provides a platform for processing and transforming continuous event streams in real-time to synchronize state and derive insights.
Deepface is a comprehensive deep learning library for facial recognition and demographic analysis. It provides a modular pipeline that handles the entire lifecycle of facial processing, including detection, geometric alignment, and the transformation of facial images into high-dimensional numerical vector embeddings for identity verification and similarity comparison. The library distinguishes itself through a model ensemble approach, which combines predictions from multiple pre-trained neural networks to improve classification accuracy and reduce bias. It also integrates advanced security fe
Processes live video feeds to perform continuous face recognition and attribute analysis.
This project is a feature-rich Go client library designed for interacting with Redis. It serves as a comprehensive interface for managing remote data stores, enabling developers to execute standard database commands, handle complex data structures, and perform asynchronous operations within Go applications. The library distinguishes itself through its support for advanced Redis capabilities, including connection pooling, pipelining, and transactional integrity. It provides specialized primitives for managing distributed clusters, including automated topology updates and request routing to sha
Supports append-only log structures for real-time event streaming and asynchronous data processing.
Vector is a high-performance observability data pipeline designed to collect, transform, and route logs, metrics, and traces across distributed infrastructure. It functions as a modular engine that decouples data ingestion from processing and transmission, utilizing a component-based architecture to connect diverse sources to multiple destinations. The project distinguishes itself through a focus on reliability and flow control. It implements backpressure-aware data movement to prevent data loss during traffic spikes and utilizes disk-backed event buffering to ensure durability during network
Routes processed logs and metrics into message topics for real-time streaming.
This project is a comprehensive framework for building AI-powered applications, providing a unified toolkit for orchestrating language models, autonomous agents, and interactive user interfaces. It serves as a central library for managing the entire lifecycle of AI interactions, from initial prompt generation and model provider abstraction to complex, multi-step reasoning and tool execution. The framework distinguishes itself through its deep integration with frontend development, specifically by enabling generative user interfaces that render dynamic components directly from model outputs. I
Transmits raw data and text tokens in real time to provide immediate feedback during generation.
This project is a comprehensive educational guide and framework for building web scrapers using Python. It provides a course-based approach to data extraction, combining a Python crawler framework with tutorials on web reverse engineering and network traffic analysis. The project distinguishes itself by covering advanced extraction challenges, including the decryption of obfuscated JavaScript and the bypass of anti-scraping measures. It specifically addresses mobile application scraping through the simulation of user interactions and the interception of network traffic. The capability surfac
Supports capturing rapidly changing information via persistent socket connections for live data streaming.