sourcegraph/conc
Conc
conc is a Go concurrency library and structured concurrency framework providing primitives for managing parallel tasks, mapping slices, and collecting results. It implements a system for spawning scoped tasks to ensure all child processes complete before their parent exits.
The library includes a goroutine pool manager to limit active concurrent processes and a panic-safe task runner that catches panics in goroutines and propagates stack traces to the parent. It also provides a concurrent map-reduce tool for transforming data slices and processing streams in parallel while maintaining the original order of results.
The project covers broad concurrency capabilities, including parallel data processing, resource-limited task execution, and structured error handling for capturing and propagating failures from parallel processes.
Features
- Concurrency Libraries - Implements a comprehensive set of Go primitives for managing parallel tasks and collecting results.
- Scoped Task Synchronization - Implements a structured concurrency framework ensuring all child work completes before the parent exits to prevent resource leaks.
- Parallel Mappers - Transforms each element of a list using a function executed across multiple parallel processes.
- Parallel Data Iterators - Splits data collections into individual units of work for concurrent processing across multiple Go routines.
- Parallel Data Transformation - Transforms or iterates over large slices of data using multiple processes to improve performance.
- Parallel Map-Reduce Tools - Provides a concurrent map-reduce tool for transforming data slices and processing streams in parallel.
- Go Concurrency Utilities - Provides a specialized set of primitives for managing Go routines with scoped ownership and resource safety.
- Structured Concurrency Managers - Provides a structured concurrency framework for spawning scoped tasks that ensures all child processes complete before the parent exits.
- Goroutine Pools - Provides a goroutine pool manager to limit active concurrent processes and prevent system resource exhaustion.
- Concurrent Task Runners - Runs multiple functions in parallel and gathers their results or errors into a single collection.
- Concurrent Task Limiters - Controls resource consumption by executing concurrent tasks using a fixed maximum number of processes.
- Panic Propagation - Captures failures in scoped processes and sends them to the parent to preserve debugging stack traces.
- Parallel Task Spawning - Runs a collection of functions concurrently and aggregates results into a single data structure.
- Panic Recovery - Captures runtime panics in child goroutines and re-throws them in the parent to preserve original stack traces.
- Stream Processing - Performs asynchronous operations on data streams in parallel while maintaining the original sequence of results.
- Concurrent Data Collection - Gathers return values and errors from multiple parallel tasks into a single unified collection.
- Thread-Safe Communication Channels - Implements thread-safe communication channels to aggregate results and errors from parallel processes into a shared slice.
- Synchronization Primitives - Utilizes synchronization primitives to block parent processes until all spawned child tasks have completed.
- Error Handling - Captures panics and errors from parallel processes and propagates them to a parent for debugging.
- Result Sequencers - Ensures that parallel outputs are returned in their original input sequence by tracking indices.
- Concurrent - Includes a panic-safe task runner that catches goroutine panics and propagates stack traces to the parent.
- Task Scheduling and Queues - Structured concurrency control.
- Concurrency Management - Provides structured concurrency tools for safer task execution.
- Concurrency Tools - Listed in the “Concurrency Tools” section of the Awesome Go awesome list.
Star history
How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.
