14 repositorios
Concurrent execution of multiple shell commands to reduce total workflow latency.
Distinguishing note: Shortlist focuses on script-specific runtimes (PHP, JS) or crawlers, not general parallel shell process management.
Explore 14 awesome GitHub repositories matching development tools & productivity · Parallel Command Execution. Refine with filters or upvote what's useful.
Lefthook es un gestor de Git hooks y automatizador de flujos de trabajo que utiliza un enfoque basado en configuración para gestionar scripts activados por eventos de control de versiones. Funciona como un ejecutor de tareas en paralelo, ejecutando múltiples scripts simultáneamente para reducir el tiempo requerido para las comprobaciones de pre-commit o pre-push. El proyecto proporciona un ejecutor de hooks contenedorizado para garantizar dependencias y conjuntos de herramientas consistentes en diferentes máquinas de desarrolladores. También admite la orquestación de monorepos ejecutando scripts dentro de subdirectorios específicos y gestionando estructuras de proyectos complejas. El sistema incluye capacidades para el filtrado de archivos basado en patrones para ejecutar tareas de control de calidad solo en activos modificados. Además, admite la ejecución manual de tareas, el etiquetado y agrupación de comandos, y anulaciones de configuración local para ajustes específicos del entorno.
Provides concurrent execution of multiple shell commands to minimize the time spent waiting for git hook completion.
Lefthook is a Git hook manager and workflow automation tool designed to execute user-defined scripts during version control events. It automates the execution of linters, tests, and formatters to validate changes and maintain code quality before commits or pushes. The project distinguishes itself through a parallel execution model that runs multiple shell commands concurrently to reduce latency. It supports complex configuration management, including the ability to merge global, remote, and local settings and synchronize configurations from external repositories to standardize development env
Executes multiple commands and scripts concurrently to significantly reduce total execution time.
k3sup is a command-line tool that installs and manages lightweight Kubernetes clusters entirely over SSH, without requiring any pre-installed software on the target machines. It bootstraps clusters on remote hosts using a single statically-linked binary, then provides immediate kubectl access by fetching and merging the cluster's admin credentials into the local configuration. The tool supports plan-driven cluster topology, allowing users to define node roles and configuration in a YAML file for automated multi-node deployment. The tool distinguishes itself by enabling parallel execution of i
Runs installation and management commands concurrently across all specified hosts.
Pixie is an open-source observability platform for Kubernetes that uses eBPF to automatically capture telemetry data from clusters without requiring any manual instrumentation or code changes. It functions as an eBPF telemetry collector, a continuous application profiler, a network traffic analyzer, and a scriptable telemetry query engine, all within a single Kubernetes-native tool. The platform distinguishes itself through several integrated capabilities. It continuously samples stack traces from compiled-language code to identify CPU performance bottlenecks, visualizing the results as inter
Runs bpftrace scripts on all cluster nodes and makes the output queryable and visualizable in the UI.
This is a GitHub Action that executes shell commands on remote servers over SSH as part of a CI/CD pipeline. It supports authentication using passwords, private keys, or private keys with passphrases, and can verify the SHA256 fingerprint of the remote host's public key before connecting to prevent man-in-the-middle attacks. The action can connect to multiple remote hosts in a single workflow step, optionally executing commands synchronously across all of them. It also supports routing SSH connections through an intermediate jump host to reach target servers in restricted networks. Environmen
Connects to multiple remote hosts simultaneously and executes commands across all of them in one step.
Pueue is a task queue manager for shell commands, built as a daemon and command-line interface. It accepts shell commands into a managed queue and executes them with configurable parallel execution limits, supporting both global and per-group concurrency controls. The daemon persists its entire state—task queue, logs, and configuration—to disk, ensuring survival across crashes and system restarts. The project distinguishes itself through a dependency graph that lets tasks declare prerequisites, forming a directed acyclic graph that controls execution order. Tasks can be organized into named g
Queues shell commands and runs them concurrently with configurable per-group parallelism limits.
ffmpeg-kit is a cross-platform SDK that wraps FFmpeg and FFprobe into native libraries for Android, iOS, macOS, Linux, and tvOS, enabling applications to execute media processing commands through platform-specific APIs. It provides a concurrent command executor that runs multiple FFmpeg operations simultaneously and collects results independently via thread-safe interfaces. The project includes a build system that compiles FFmpeg native libraries from source with configurable codec and library options for each target platform, and offers eight precompiled binary packages with different sets o
Runs multiple FFmpeg commands simultaneously and collects independent results per command via thread-safe APIs.
pyinfra is an agentless infrastructure automation framework that turns declarative Python code into idempotent shell commands to manage servers, containers, and local machines over SSH without requiring any pre-installed software on target hosts. It operates by comparing the desired state of a system against its current state, using a dry-run simulation mode to preview changes and a fact-based conditional execution engine to gather host attributes at runtime and control which operations run. The tool compiles Python operations into optimized shell commands and executes them in parallel across
Runs operations concurrently across thousands of hosts with configurable parallelism.
pyinfra is a Python-based infrastructure automation framework that turns Python code into shell commands for managing servers, Docker containers, and local machines. It operates as a declarative, idempotent deployment tool, applying desired system states by comparing target configurations against current states and making only the necessary changes. The framework provides a connector-based transport abstraction that unifies SSH, Docker, and local execution behind a common interface, with a parallel execution engine that manages concurrent operations across hosts. The tool distinguishes itself
Manages concurrent operation execution across hosts with configurable parallelism and serialization controls.
Axiom es un orquestador de infraestructura en la nube y framework de escaneo de seguridad distribuido. Sirve como un gestor para desplegar, tomar snapshots y destruir flotas de máquinas virtuales desechables a través de múltiples proveedores de nube y regiones. El proyecto se distingue por automatizar el aprovisionamiento de conjuntos de herramientas de vulnerabilidad y software de auditoría de seguridad a través de estos servidores remotos. Cuenta con un mecanismo para distribuir escaneos de seguridad fragmentando listas de objetivos a través de una flota de instancias y agregando los datos resultantes en archivos unificados e informes HTML. El sistema cubre una amplia gama de capacidades operativas, incluyendo ejecución de comandos remotos, transferencias de archivos paralelas y ajuste de red a nivel de kernel para cargas de trabajo de alta concurrencia. También proporciona herramientas para la gestión de snapshots de imágenes, túneles SSH para tráfico de red y el despliegue de perfiles de infraestructura predefinidos.
Runs a single command across all selected remote instances simultaneously in foreground or background processes.
Mgmt es un sistema de gestión de configuración distribuido que mantiene el estado deseado de los clústeres utilizando automatización basada en eventos y retroalimentación de bucle cerrado. Funciona como un motor de automatización de infraestructura que activa correcciones de estado del sistema en tiempo real basadas en el monitoreo de recursos y especificaciones predefinidas. El sistema incluye un selector de nodos de clúster distribuido para elegir subconjuntos de hosts basados en estrategias y restricciones específicas para distribuir cargas de trabajo. También cuenta con un gestor de infraestructura en la nube para controlar el ciclo de vida de las instancias de máquinas virtuales, incluyendo despliegue de imágenes, selección de región y scripts de inicio, junto con un orquestador de firmware de hardware para instalar y verificar binarios en controladores de gestión de placa base. Las capacidades adicionales cubren la ejecución de tareas en paralelo, la modificación de archivos de configuración basada en lentes y la recuperación de metadatos del sistema. El conjunto de herramientas también proporciona utilidades para la transformación de tipos de datos, decodificación de formatos e inspección del entorno para determinar los estados actuales del sistema.
Coordinates configuration tasks across multiple remote hosts concurrently from a single workflow step.
GAM is a command-line tool for administering Google Workspace and Cloud Identity. It translates command-line arguments into structured API calls, enabling administrators to manage users, groups, organizational units, and domain settings across a Google Workspace environment. The tool handles authentication through OAuth2 flows, service accounts, and workload identity federation, and supports multi-tenant configurations for managing multiple domains or cloud projects from a single installation. GAM distinguishes itself through its batch processing and automation capabilities. It can process la
Implements parallel and threaded execution of command sequences defined in local files or cloud documents.
s5cmd is a command line interface for high-performance data transfers and management tasks across S3 compatible storage services. It functions as a parallel data transfer tool and bucket synchronization utility, designed to accelerate the uploading and downloading of large volumes of files using concurrent workers. The tool acts as a batch command processor capable of executing thousands of object management operations in parallel from command files or piped streams. It also serves as an S3 Select query client, allowing the execution of SQL expressions against stored JSON records to retrieve
Enables concurrent execution of thousands of object management tasks from command files or piped streams.
This project is a parallel test runner and shell command executor designed to reduce total runtime by executing test suites across multiple CPU cores. It functions as a test suite load balancer and a CI test process manager that coordinates global setup, teardown, and exit codes for continuous integration pipelines. The tool acts as a test database orchestrator by creating and managing unique database instances mapped to parallel processes, which prevents data collisions. It distributes tests into balanced groups based on file size or recorded runtimes to optimize execution time and avoid bot
Executes any shell command across multiple processes, assigning each a unique process identifier.