sourcegraph/conc
Conc
conc is a Go concurrency library and structured concurrency framework providing primitives for managing parallel tasks, mapping slices, and collecting results. It implements a system for spawning scoped tasks to ensure all child processes complete before their parent exits.
The library includes a goroutine pool manager to limit active concurrent processes and a panic-safe task runner that catches panics in goroutines and propagates stack traces to the parent. It also provides a concurrent map-reduce tool for transforming data slices and processing streams in parallel while maintaining the original order of results.
The project covers broad concurrency capabilities, including parallel data processing, resource-limited task execution, and structured error handling for capturing and propagating failures from parallel processes.
Features
- Concurrency Libraries - Implements a comprehensive set of Go primitives for managing parallel tasks and collecting results.
- Scoped Task Synchronization - Implements a structured concurrency framework ensuring all child work completes before the parent exits to prevent resource leaks.
- Parallel Mappers - Transforms each element of a list using a function executed across multiple parallel processes.
- Parallel Data Iterators - Splits data collections into individual units of work for concurrent processing across multiple Go routines.
- Parallel Data Transformation - Transforms or iterates over large slices of data using multiple processes to improve performance.
- Parallel Map-Reduce Tools - Provides a concurrent map-reduce tool for transforming data slices and processing streams in parallel.
- Go Concurrency Utilities - Provides a specialized set of primitives for managing Go routines with scoped ownership and resource safety.
- Structured Concurrency Managers - Provides a structured concurrency framework for spawning scoped tasks that ensures all child processes complete before the parent exits.
- Goroutine Pools - Provides a goroutine pool manager to limit active concurrent processes and prevent system resource exhaustion.
- Concurrent Task Runners - Runs multiple functions in parallel and gathers their results or errors into a single collection.
- Concurrent Task Limiters - Controls resource consumption by executing concurrent tasks using a fixed maximum number of processes.
- Panic Propagation - Captures failures in scoped processes and sends them to the parent to preserve debugging stack traces.
- Parallel Task Spawning - Runs a collection of functions concurrently and aggregates results into a single data structure.
- Panic Recovery - Captures runtime panics in child goroutines and re-throws them in the parent to preserve original stack traces.
- Stream Processing - Performs asynchronous operations on data streams in parallel while maintaining the original sequence of results.
- Concurrent Data Collection - Gathers return values and errors from multiple parallel tasks into a single unified collection.
- Thread-Safe Communication Channels - Implements thread-safe communication channels to aggregate results and errors from parallel processes into a shared slice.
- Synchronization Primitives - Utilizes synchronization primitives to block parent processes until all spawned child tasks have completed.
- Error Handling - Captures panics and errors from parallel processes and propagates them to a parent for debugging.
- Result Sequencers - Ensures that parallel outputs are returned in their original input sequence by tracking indices.
- Concurrent - Includes a panic-safe task runner that catches goroutine panics and propagates stack traces to the parent.
- Task Scheduling and Queues - Structured concurrency control.
- Concurrency Management - Provides structured concurrency tools for safer task execution.
- Concurrency Tools - Listed in the “Concurrency Tools” section of the Awesome Go awesome list.
