15 repositorios
Architectures that distribute network IO and query processing across multiple CPU cores.
Distinct from Network Request Processing: Specific to the multi-threaded execution of database queries rather than general network request lifecycles.
Explore 15 awesome GitHub repositories matching software engineering & architecture · Multi-Threaded Request Handling. Refine with filters or upvote what's useful.
KeyDB is a multithreaded in-memory key-value store and distributed cache. It functions as a NoSQL database utilizing multi-version concurrency control to execute non-blocking queries and scans. The project is a multithreaded fork of Redis that maintains protocol compatibility while utilizing a multithreaded architecture to scale across multi-core hardware. It distinguishes itself with flash-tiered storage, allowing the system to offload data from primary RAM to SSD or flash storage to increase total capacity. The system supports high availability through active-active mesh replication and mu
Distributes network IO and query parsing across multiple CPU cores to handle concurrent requests without blocking.
Garnet is a multi-threaded in-memory database and distributed key-value store. It functions as a high-performance remote cache store that implements the RESP wire protocol to maintain compatibility with existing Redis clients and libraries. The project is distinguished by a shared-memory architecture that enables parallel request processing across multiple cores for sub-millisecond latency. It features a tiered storage system that automatically offloads colder data from system memory to SSD or cloud storage layers, and includes a specialized vector search database for high-dimensional similar
Distributes network IO and query processing across multiple CPU cores to maintain high throughput and low latency.
Webmagic is a Java web crawling framework designed for building scalable automated crawlers to download and process large volumes of web pages. It functions as a distributed web crawler and dynamic content crawler, utilizing an XPath HTML parser to locate and extract specific data points from page structures. The framework distinguishes itself through its ability to handle dynamic content by rendering JavaScript and executing asynchronous requests to extract data from non-static pages. It also allows users to define and execute crawler logic via scripting languages, enabling the update of col
Distributes web requests and page parsing across multiple concurrent threads to increase total data collection throughput.
This project is a command-line utility and Python-based scraper designed to download and archive educational media from Coursera for offline access. It functions as a tool for saving course syllabi, videos, subtitles, and notebooks to a local file system to ensure a secure backup of academic materials. The tool distinguishes itself by providing capabilities for course quiz extraction, retrieving questions and answers from both graded and ungraded quizzes. It supports the retrieval of learning materials from both legacy and on-demand platforms and implements interrupted download resumption to
Distributes network requests across multiple threads to increase throughput and optimize download performance.
Puma is a concurrent HTTP server for Ruby applications that implements the Rack interface. It operates as a clustered web server, using a combination of worker processes and threads to handle multiple simultaneous web connections via TCP ports or UNIX domain sockets. The server features a master-worker process model that utilizes multiple CPU cores and employs copy-on-write preloading to reduce memory usage. It supports zero-downtime restarts through socket-handover capabilities, allowing application updates without dropping pending network requests. The project includes a token-authenticate
Scales web traffic by distributing network IO and request handling across multiple CPU cores using threads and processes.
Robyn is a high-performance asynchronous web framework and server for Python that utilizes a Rust-powered runtime to execute code. It functions as an asynchronous Python web server designed to handle many concurrent requests efficiently through a multithreaded execution model. The project includes specialized capabilities for AI agent request routing, managing communication between agents and external tools. It also serves as a WebSocket communication server, maintaining persistent bidirectional channels for real-time data exchange. The framework covers a broad range of web infrastructure, i
Distributes network traffic across multiple operating system threads to process concurrent requests in parallel.
This project is a proxy aggregation platform designed to collect and verify free proxy server lists from web platforms, social media, and public repositories. It functions as a crawler framework that gathers proxy data and subscription links, a validation tool for testing server liveness, and a synchronization service for distributing the results. The system uses a plugin-based architecture that allows for the integration of custom Python scripts to handle diverse web source structures. It also includes utilities to transform raw proxy data into standardized configuration formats compatible w
Utilizes multi-threaded execution to increase the throughput of high-volume proxy data collection and validation.
CocoaHTTPServer es un servidor web embebido ligero diseñado para manejar solicitudes de red entrantes y servir contenido de aplicaciones locales. El proyecto implementa el descubrimiento de servicios basado en Bonjour para transmitir la disponibilidad del servidor en redes locales y admite WebDAV para la gestión remota de archivos, permitiendo que los archivos sean creados, movidos y editados a través de HTTP. La seguridad de la red se maneja a través de cifrado TLS basado en certificados y autenticación de recursos basada en contraseña. El enrutamiento de solicitudes se gestiona a través de un sistema basado en delegados, mientras que las conexiones concurrentes se manejan utilizando Grand Central Dispatch.
Uses Grand Central Dispatch to manage concurrent network connections via a pool of worker threads.
MiniOB is an open-source educational relational database kernel designed for learning the internals of database systems. It implements a dual-engine storage architecture combining B+ Tree and LSM-Tree, supports SQL parsing and query execution, and provides transactional processing with multi-version concurrency control. The system communicates with clients using the MySQL wire protocol and includes a vector database extension for storing and querying high-dimensional vectors. The project distinguishes itself through its comprehensive coverage of core database concepts in a single, learnable c
Assign each client connection its own thread that listens for messages, processes SQL, and sends responses.
Este proyecto es un servidor web HTTP en C++ diseñado para el entorno Linux. Funciona como un servidor de red de alto rendimiento que procesa peticiones HTTP y entrega archivos estáticos. El servidor utiliza un modelo de reactor y un pool de hilos para manejar grandes volúmenes de conexiones simultáneas. Emplea una máquina de estados y expresiones regulares para el análisis de peticiones, mientras que un sistema de temporizador basado en min-heap gestiona los tiempos de espera de las conexiones para cerrar automáticamente los sockets inactivos. Otras capacidades incluyen un pool de conexiones a bases de datos para reducir la sobrecarga del handshake y un sistema integrado de autenticación de usuarios para registro e inicio de sesión. La observabilidad del sistema se proporciona a través de un sistema de logging asíncrono que registra eventos en tiempo de ejecución en segundo plano.
Implements a reactor model and thread pool to distribute network IO and request processing across multiple CPU cores.
SocialBox-Termux is a credential bruteforce suite and security tool collection designed to run within the Termux Android environment. It functions as an automated account cracker used to test password lists against usernames to discover valid login credentials for social media and email platforms. The toolkit incorporates network traffic masking by routing requests through the Tor network to conceal the origin IP address. It utilizes signature-based rate bypass to mimic legitimate client traffic and avoid automated login blocks. The suite provides capabilities for account validation to verif
Implements multi-threaded network request handling to accelerate the process of testing password lists.
Swifter is a lightweight HTTP server engine and networking library for the Swift programming language. It provides a minimal networking layer designed for hosting internal web services, building server-side applications, and implementing custom HTTP servers. The project enables the creation of embedded web services for remote control or status monitoring and supports lightweight web hosting with a minimal footprint. Its capabilities cover the management of raw TCP connections and network sockets, using a request-response cycle to parse incoming data into HTTP requests and map them to handler
Processes incoming HTTP requests sequentially on a blocking thread.
sourcekit-lsp is a language server protocol implementation and IDE code intelligence backend. It functions as a Swift language server and source code indexer, providing semantic analysis, autocomplete, and navigation features to compatible text editors. The project integrates directly with the Swift toolchain and SourceKit framework to deliver precise type information. It also leverages Clang-integrated parsing to provide semantic analysis for C-family languages, enabling cross-language navigation across project files. The server manages source code indexing and symbol lookups while using a
Implements non-blocking request handling to ensure the text editor remains responsive during heavy indexing tasks.
This project is a collection of interface components designed to enhance the Language Server Protocol experience within the Neovim text editor. It provides a suite of modular visual elements that allow developers to interact with language server data through floating windows, improving the presentation and accessibility of code information directly within the editor workspace. The plugin distinguishes itself by offering specialized UI compositions for complex development tasks, such as visualizing call hierarchies, managing diagnostic issues, and performing global symbol renaming with live pr
Handles language server queries asynchronously to maintain editor responsiveness during complex data retrieval.
NFS-Ganesha es un servidor de archivos en modo usuario que proporciona acceso de red a recursos de almacenamiento utilizando los protocolos NFSv3, v4 y v4.1. Al operar completamente en el espacio de usuario, funciona como una puerta de enlace de almacenamiento que permite el intercambio remoto de archivos entre sistemas Linux y Unix mientras evita las complejidades y riesgos de estabilidad asociados con el desarrollo de sistemas de archivos a nivel de kernel. El proyecto se distingue por una abstracción de sistema de archivos modular que le permite traducir solicitudes de red en operaciones para diversos sistemas de almacenamiento backend. Mantiene un manejo de protocolo con estado para admitir funciones avanzadas como el bloqueo de archivos y delegaciones, mientras utiliza un modelo de despacho de solicitudes multihilo y un núcleo basado en eventos asíncronos para gestionar el tráfico concurrente en hardware multinúcleo. Más allá de sus capacidades principales de servidor, el software admite la gestión dinámica de configuración, permitiendo a los administradores actualizar exportaciones y políticas de seguridad en tiempo de ejecución sin interrumpir las conexiones activas de los clientes. Sirve como una interfaz unificada para la integración de almacenamiento distribuido, facilitando el acceso a archivos multiplataforma y la gestión centralizada de directorios compartidos a través de una red.
Distributes incoming network traffic across a pool of worker threads to maximize performance on multi-core server hardware.