15 repository-uri
Architectural patterns for processing user actions through validation and execution stages.
Distinguishing note: Focuses on declarative action handling.
Explore 15 awesome GitHub repositories matching software engineering & architecture · Command Pipelines. Refine with filters or upvote what's useful.
This project is a feature-rich Go client library designed for interacting with Redis. It serves as a comprehensive interface for managing remote data stores, enabling developers to execute standard database commands, handle complex data structures, and perform asynchronous operations within Go applications. The library distinguishes itself through its support for advanced Redis capabilities, including connection pooling, pipelining, and transactional integrity. It provides specialized primitives for managing distributed clusters, including automated topology updates and request routing to sha
Sends batches of requests to the server without waiting for individual responses to reduce network round-trip time.
This project is a database driver for Node.js applications designed to interface with Redis. It provides structured access to data stores, enabling the execution of commands, management of data structures, and the implementation of atomic transaction processing. The client distinguishes itself through native support for the binary-safe serialization protocol and a promise-based command pipeline that groups operations to minimize latency. It includes a dedicated manager for distributed environments that handles node discovery and request routing, alongside an event-driven messaging system that
Groups multiple asynchronous operations into single network requests to minimize latency and maximize throughput.
Memcached is a high-performance, distributed, in-memory key-value storage and request routing engine. It functions as a volatile data store designed to accelerate dynamic applications by caching objects in RAM, thereby reducing backend database load and providing sub-millisecond response times. The system utilizes a specialized architecture that organizes memory into fixed-size slabs to minimize fragmentation and maximize throughput for high-concurrency workloads. The project distinguishes itself through a multi-threaded, lock-friendly design that scales across CPU cores and supports complex
Batches multiple commands into single network streams using tokens to track and match responses efficiently.
Twemproxy is a lightweight proxy that routes and distributes requests across multiple Redis and Memcached backend servers. It functions as a protocol translation gateway and distributed cache shard manager, partitioning data across clusters to balance load and storage capacity. The system acts as a high-availability cache orchestrator, employing health monitoring and automatic server ejection to maintain continuous access to cached data. It integrates with sentinels for dynamic master and replica discovery and utilizes consistent hashing and tag-based key grouping to manage data distribution
Batches multiple client requests into a single network message to reduce round-trip time and increase throughput.
obsidian-git is a version control integration for personal knowledge management that enables local file versioning and synchronization. It provides Git-based tracking for local files to allow for history auditing, document reversion, and the backup of notes to remote servers. The project automates the version control process through scheduled background syncing, which triggers automatic commits and pushes at set intervals or during application startup. It facilitates collaborative editing by using branches and synchronization to manage contributions and merge changes. The toolset covers comp
Implements a sequential pipeline to ensure multiple file changes are committed atomically during synchronization.
phpredis is a C-based native extension that bridges PHP applications with Redis servers for high-performance data storage and retrieval. It serves as an interface for manipulating strings, hashes, lists, sets, and sorted sets while providing a direct path for executing Redis commands and server-side scripts. The extension provides comprehensive support for distributed environments and high availability. It interfaces with Redis Cluster to distribute data across multiple nodes using hash slots and manages Redis Sentinel for service discovery and automatic failover. It also enables shared state
Implements socket-level command pipelining to group multiple requests into a single network write.
RTranslator is a speech translation application and background speech processor designed for real-time voice and text translation. It functions as a dual-language audio translator that detects different spoken languages to provide immediate voice translations for face-to-face interactions. The system includes a multi-device translation setup that synchronizes multiple devices over a network to facilitate group conversations through shared speakers. It also provides the ability to maintain audio processing and speech translation tasks while the device is on standby. The project covers a range
Implements an asynchronous pipeline to handle text and speech conversion without blocking the user interface.
Japronto is an asynchronous web framework and Python HTTP server toolkit. It functions as a multi-worker HTTP server and request router, utilizing non-blocking asynchronous handlers to manage high concurrency and throughput. The project implements a master-multiworker forking model to distribute network traffic across multiple CPU cores. It incorporates a fast event loop and a specialized C-extension for high-speed HTTP request parsing, while supporting request pipelining over single TCP connections. The framework covers a broad range of request handling capabilities, including URL pattern r
Supports HTTP pipelining, allowing multiple requests to be sent over a single TCP connection without waiting for responses.
Undici is a high-performance HTTP client for Node.js designed to execute network requests while minimizing resource overhead and latency. The project provides specialized capabilities for HTTP request pipelining to increase throughput and memory-based response caching to reduce redundant network calls. It also supports HTTP tunneling and proxying through the establishment of secure two-way communication channels via the connect method. Additionally, the client includes tools for web API polyfilling to provide standard browser network interfaces within Node.js environments. Its functional sur
Increases data throughput by pipelining multiple HTTP requests over a single connection.
JMComic-Crawler-Python is a high-performance asynchronous web scraper and API client designed to programmatically retrieve images and metadata from a comic hosting service. It functions as a media archiving tool for batch downloading albums and chapters, automating the process of saving content to a local filesystem. The project is distinguished by its ability to reverse server-side pixel obfuscation, using a decryption tool to reconstruct sliced and shuffled images. To maintain stable connectivity, it utilizes a network bypass utility featuring dynamic domain rotation and proxy routing to ci
Implements an asynchronous pipeline that separates network I/O from decryption to maximize data throughput.
General purpose redis client
Queues commands and processes responses asynchronously over a single TCP connection for high throughput.
Blackbird is an open-source OSINT investigation tool that searches across hundreds of online platforms to discover accounts linked to a given username or email address. It functions as a username and email search engine, consolidating discovered profiles into a single list with low false positives for investigative analysis. The tool incorporates an AI-enhanced profile analyzer that uses a built-in AI API to generate behavioral and technical summaries of discovered online profiles. It also provides a documentation query interface that accepts natural-language questions via HTTP GET requests t
Implements concurrent HTTP requests across platform adapters for efficient search fan-out.
lakeFS is a data lake versioning system that provides Git-like branching and commits for large datasets stored in object storage. It functions as a version control layer, enabling the creation of immutable snapshots, atomic commits, and zero-copy branching to create isolated environments for data experimentation without duplicating physical files. The system serves as an S3-compatible storage gateway and an Iceberg REST catalog, allowing standard cloud storage protocols and compatible clients to manage versioned tables. It acts as a data quality gatekeeper by using an event-driven hook system
Groups multiple object changes into a single immutable snapshot to ensure data consistency and reproducibility.
This project is a DNS privacy proxy and resolver that functions as a local bridge, converting plaintext DNS traffic into encrypted requests. It acts as a client for DNS-over-HTTPS and DNS-over-TLS protocols to prevent interception and spoofing of network requests. The system implements network privacy hardening by routing domain lookups through secure tunnels, which reduces the amount of plain text data leaked to internet service providers. It utilizes a profile-based connection management system to map security profiles to specific encrypted endpoints, preventing DNS hijacking and man-in-the
Processes multiple DNS queries concurrently using non-blocking I/O to maximize throughput.
Rueidis is a high-performance Redis client library for Go that provides a type-safe and asynchronous interface for interacting with Redis servers. It includes a full implementation of the Redis serialization protocol and a dedicated connection manager to handle pooling, multiplexing, and automatic pipelining. The library is distinguished by its support for RDMA connectivity to reduce latency and CPU overhead. It features a distributed lock manager that implements majority-based locking and optimistic concurrency control, as well as client-side caching with invalidation signals to minimize net
Supports manual batching of multiple database commands into a single network request.