30 open-source projects similar to sourcegraph/conc, ranked by how many features they have in common. Compare stars, activity and what each one does to find the best Conc alternative.
Ants is a goroutine pool library for Go that functions as a concurrent task orchestrator and resource consumption limiter. Its primary purpose is to manage and reuse a pool of goroutines to limit concurrency and reduce the memory allocation overhead associated with frequent thread creation. The system provides a runtime worker manager capable of adjusting pool capacity dynamically to respond to fluctuating workloads. To maintain stability, it includes panic recovery mechanisms that intercept runtime failures within worker threads to prevent a single failing task from crashing the entire appli
Rayon is a data parallelism library for Rust that provides a framework for converting sequential computations into parallel operations. It enables the transformation of standard data structures and loops into parallel iterators, allowing workloads to be distributed across multiple processor cores. By utilizing a work-stealing scheduler, the library dynamically balances tasks to maximize throughput and minimize execution time. The library distinguishes itself through its focus on safe, scoped task synchronization, which ensures that all spawned operations complete before a scope exits to preve
Tunny is a worker pool library and concurrency manager for Go. It functions as a concurrent task scheduler that limits the number of parallel workers processing jobs to prevent system resource exhaustion. The project enables the maintenance of persistent state within individual worker routines to avoid repeated initialization costs. It also supports dynamic worker scaling, allowing the number of active background workers to be adjusted in real time without interrupting tasks in progress. The library provides mechanisms for task timeout enforcement and worker lifecycle management, including t
Trio is an asynchronous I/O runtime and concurrency library for Python. It provides a system for executing non-blocking network and disk operations through a centralized event loop and task scheduler. The library is built on a structured concurrency model, which ensures that asynchronous tasks are bound to a specific lifetime and cannot outlive the scope that started them. It utilizes a nursery-based task manager to track task lifecycles in a parent-child tree, preventing orphaned concurrent operations by requiring child tasks to be joined before their parent scope exits. The framework cover
Concurrent Ruby is a comprehensive concurrency toolkit for the Ruby language that provides thread-safe data structures, synchronization primitives, and asynchronous execution patterns. It implements core concurrency abstractions including an actor model framework where isolated actors communicate through asynchronous message passing, a future and promise system for composing non-blocking operations, and thread pool executors that manage reusable worker threads for concurrent task execution. The library distinguishes itself through a broad set of coordination mechanisms that go beyond basic th
This project is a comprehensive performance programming guide and reference for the Go language, focusing on runtime efficiency and memory optimization. It provides a collection of patterns and techniques designed to increase execution speed by reducing garbage collection overhead and optimizing memory usage. The resource distinguishes itself through detailed reference implementations for memory optimization, such as escape analysis, object pooling, and structure memory alignment. It offers specific strategies for reducing binary size and improving CPU cache efficiency through structure memor
Crystal is a statically typed, compiled programming language designed for high performance and memory safety. It leverages an LLVM-based compiler to translate source code into optimized machine-executable binaries, while its type-inference-based static analysis enforces strict safety rules during the build process. The language distinguishes itself through a fiber-based concurrent runtime that manages lightweight execution units for asynchronous input and output without blocking the main process. It also features a powerful compile-time macro system that allows for the inspection and transfor
The Rust Programming Language Book is the official technical guide and educational resource for the Rust language. It provides a comprehensive walkthrough of the language's design, focusing on its core identity as a systems programming language that enforces memory safety and high-performance execution without the need for a garbage collector. The project is distinguished by its focus on ownership, borrowing, and lifetime tracking, which allow the compiler to verify memory safety and thread safety at compile time. It covers the language's unique approach to zero-cost abstractions, including t
Asynq is a distributed background job processing framework for Go applications. It manages asynchronous task queues by offloading heavy operations to persistent storage, allowing the main application to remain responsive while background workers handle workloads. The system utilizes Redis to manage task state, concurrency, and message distribution across multiple worker instances. It employs atomic Lua scripting and sorted sets to ensure reliable job acquisition, precise scheduling of delayed tasks, and fault-tolerant processing through a two-stage acknowledgement flow. The framework support
Python is a high-level, interpreted programming language designed for readability and versatility. It operates via a bytecode-based virtual machine and manages memory automatically through reference-counting garbage collection. The language supports multiple programming paradigms, including object-oriented, imperative, and functional styles, and provides a comprehensive standard library for system operations, networking, and data handling. The language is distinguished by its dynamic nature, allowing for runtime object introspection and metaclass-driven class creation. It utilizes protocol-ba
Concurrency limiting goroutine pool
🔘 Minimalistic and High-performance goroutine worker pool written in Go
Async is a JavaScript asynchronous flow library designed to manage the execution and coordination of asynchronous tasks in Node.js and the browser. It provides functional utilities to wrap, process, and orchestrate complex asynchronous workflows. The library distinguishes itself through a comprehensive task orchestrator that handles dependency graphs to resolve circular references and manages concurrent task queues. It includes a unification bridge that allows callback-style and promise-based functions to operate within the same execution interface. The project covers several primary capabil
Hatchet is an open-source durable workflow engine and task orchestration platform. It provides a framework for building and executing fault-tolerant, multi-step pipelines as directed acyclic graphs (DAGs), with automatic retries, scheduling, and real-time observability. The system is built around durable task checkpointing, which persists execution state after each step so work can resume from the last checkpoint after a worker crash or restart, and it supports event-driven task resumption that pauses a task until a matching external event arrives. The platform distinguishes itself through it
Kotlinx.coroutines is a library for managing non-blocking background tasks and structured concurrency within the Kotlin programming language. It provides a framework for executing concurrent operations and synchronizing shared state, replacing traditional thread management and complex callback chains with lightweight primitives. The library utilizes a structured concurrency hierarchy to organize hierarchical background tasks, ensuring that lifecycle management, cancellation, and timeout handling propagate automatically to prevent resource leaks. It employs continuation-passing style transform
Gevent is a Python coroutine concurrency library and asynchronous task manager designed for high-concurrency I/O tasks. It provides a cooperative networking framework for building asynchronous TCP, UDP, and HTTP servers, as well as a WSGI web server implementation for hosting web applications. The project is distinguished by its standard library monkey-patching tool, which replaces blocking synchronous functions with cooperative versions to enable asynchronous behavior in third-party code. This allows for a cooperative multitasking workflow where the system yields execution during I/O waits t
c3c is the compiler for the C3 programming language, transforming source code into executable binaries, static libraries, or dynamic libraries using an LLVM backend. It implements a system based on result-based error handling, scoped memory pooling, and a semantic macro system. The compiler provides first-class support for hardware-backed SIMD vectors that map directly to processor instructions and enables runtime polymorphism through interface-based dynamic dispatch. The project covers a broad set of low-level capabilities, including manual and pooled memory management, inline assembly inte
Oils is a Unix shell interpreter and scripting language runtime that combines a modern shell language with POSIX and Bash compatibility. It functions as a structured data shell, integrating JSON and J8 formats to manage complex data without relying on string parsing. The system is distinguished by a C++ transpiled interpreter that converts a subset of Python source code into strongly typed C++ for high-performance execution. It features a garbage-collected typed runtime that supports prototype-based polymorphism, lexical closures, and reference-based variable mutation. To decouple shell logic
gocron is a task scheduling library for Go designed to automate recurring tasks using crontab expressions, fixed durations, and specific calendar dates. It functions as a concurrent job manager and execution monitor that tracks task lifecycles and performance. The project provides distributed task scheduling through leader election and distributed locking to prevent duplicate job execution across multiple nodes. It further distinguishes itself with a concurrency management system that implements singleton constraints and global execution quotas to protect system resources. The scheduler cove
Machinery is a distributed task queue and asynchronous workflow engine. It provides a system for processing heavy workloads outside the main request flow using a network of distributed background workers and a message-based job orchestrator. The project manages complex task lifecycles through sequential chaining, where results are passed between tasks, and parallel coordination, which can trigger callback tasks upon the completion of a group. It supports periodic workflow scheduling for recurring jobs and delayed execution via specific timestamps. The system includes capabilities for result
RxGo is a functional reactive programming library and an implementation of ReactiveX for the Go language. It serves as an asynchronous stream processing toolkit designed to coordinate event-based programs and data flows using the observable pattern. The library enables the construction of asynchronous processing pipelines that transform, filter, and combine event sequences. It distinguishes itself through the use of functional operators to compose these pipelines and provides mechanisms for managing concurrent execution. The toolkit covers a broad range of stream orchestration capabilities,
Vector is a high-performance observability data pipeline designed to collect, transform, and route logs, metrics, and traces across distributed infrastructure. It functions as a modular engine that decouples data ingestion from processing and transmission, utilizing a component-based architecture to connect diverse sources to multiple destinations. The project distinguishes itself through a focus on reliability and flow control. It implements backpressure-aware data movement to prevent data loss during traffic spikes and utilizes disk-backed event buffering to ensure durability during network
Luigi is a Python framework designed for building and managing complex batch data pipelines. It functions as a workflow orchestration engine that organizes tasks into directed acyclic graphs, ensuring that jobs execute in the correct logical order based on their dependencies. By utilizing a centralized scheduler, the system coordinates task execution across distributed environments, tracks global workflow state, and prevents redundant processing by verifying the existence of output targets before triggering any work. The project distinguishes itself through a robust state-tracking mechanism t
simdjson is a high-performance, header-only C++ library designed for parsing, querying, and serializing JSON data with minimal memory overhead. It functions as a hardware-aware data processing engine that leverages vector instructions to achieve gigabyte-per-second parsing speeds. By detecting host processor capabilities at runtime, the library automatically selects the most efficient instruction sets to accelerate structural analysis and validation. The library distinguishes itself through a focus on extreme efficiency and resource management. It utilizes memory mapping and padded buffer ali
:speedboat: a limited consumer goroutine or unlimited goroutine pool for easier goroutine handling and cancellation
Simply way to control goroutines execution order based on dependencies
Library to manage application lifetime in a Go microservice architecture
goworker is a Go-based background worker that runs 10 to 100,000* times faster than Ruby-based workers.
🚧 Flexible mechanism to make execution flow interruptible.