HariSekhon/DevOps-Bash-tools
DevOps Bash Tools
DevOps-Bash-tools is a collection of shell scripts and aliases designed to automate cloud infrastructure, container orchestration, and CI/CD pipelines. It provides a comprehensive toolset for managing operational workflows through the command line.
The project specializes in automating tasks across multiple platforms, including managing namespaces and secrets in Kubernetes, auditing resources in AWS and GCP, and triggering builds or managing environment variables in GitHub Actions, GitLab CI, and CircleCI. It also includes a toolkit for interacting with container registries to query manifests and optimize image sizes, as well as utilities for batch processing Git repositories and enforcing commit standards.
Beyond cloud and pipeline management, the toolset covers a broad range of capabilities including system administration, development environment setup, and security auditing for identity permissions and secret leakage. It also provides utilities for media manipulation, data processing, and the automation of language runtime installations.
Features
- Bash DevOps Toolsets - Offers a comprehensive collection of shell scripts for automating cloud infrastructure and container orchestration.
- Container Orchestration Debugging - Dumps logs and system statistics from Kubernetes pods and nodes to troubleshoot applications.
- Development Environment Setup - Standardizes development environments by automating the installation of language runtimes, cloud CLIs, and shell configurations.
- Repository Synchronization - Clones or pulls multiple repositories simultaneously to synchronize code across environments.
- API Client Wrappers - Executes HTTP requests using curl and jq to automate external service management via the command line.
- Shell Workflow Optimizers - Configure shell files and aliases to streamline cloud provider workflows.
- Git Command Enhancements - Simplifies complex Git operations through short-hand aliases and command wrappers.
- Git Workflow Automation - Automates complex Git command sequences, including multi-remote synchronization and push statistics tracking.
- Repository Management - Executes administrative scripts across multiple repositories to automate repetitive tasks.
- Automated Software Delivery - Implements automation for build and test stages via configuration files to streamline software delivery.
- CI CD Pipelines - Includes tools to trigger builds and update environment variables by interacting with various CI/CD pipeline APIs.
- CI/CD Pipeline Configurations - Automates build triggers, environment variables, and workflow management for GitHub Actions, GitLab CI, and CircleCI.
- Cloud Console Authentication - Automates the cloud SSO login process by triggering requests when active sessions are missing.
- Cloud Infrastructure Automation - Executes administrative tasks across multiple cloud projects using loop-based shell scripts.
- Cloud Infrastructure Management - Audits resources, manages service accounts, and configures networking across AWS and GCP via shell scripts.
- Client Cluster Contexts - Isolates contexts and applies manifests to manage cluster-wide secrets and configuration differences.
- Container Registry Management - Lists images, manages tags, and retrieves manifests by querying container registries via API.
- Kubernetes Operations Automation - Automates routine cluster management tasks, including namespace switching and API endpoint retrieval.
- Kubernetes Cluster Management - Provides tools for managing Kubernetes namespaces, secrets, and pod resources via the command line.
- Multi-Account Batch Execution - Allows executing specific commands across every account or zone defined in a profile.
- Namespace Operations - Creates namespaces, restarts deployments, and deletes stuck resources across multiple contexts.
- Commit Checkers - Enforces commit standards by validating files against linting and formatting rules before commits.
- Pull Request Management - Automates the creation of pull requests and merging of branches to streamline code review.
- Service Port Forwarding - Simplifies access to internal cluster services by streamlining port forwarding and pod IP retrieval.
- Client Context Switching - Provides utilities to isolate and swap active Kubernetes namespaces and cloud profiles using temporary configuration files.
- Cloud Credential Management - Loads session tokens into environment variables to simplify CLI authentication for cloud platforms.
- Cloud Service Authentication - Provides utilities to load cloud credentials from environment variables to simplify authentication with cloud platforms and API services.
- Cluster Secret Manipulators - Provides utilities to convert key-value pairs into cluster secrets and manage their annotations.
- Secrets Scanning - Implements scanning for sensitive credentials in files to block them from being committed to version control.
- Kubernetes Context Isolation - Provides the ability to switch active Kubernetes namespaces for specific shell sessions using temporary config files.
- Cluster Health Monitoring - Inspects container counts and pod distributions to analyze the health of Kubernetes clusters.
- Kubernetes Administration Scripts - Provides specialized scripts for managing Kubernetes contexts, namespaces, and pod debugging.
- Container Image Storage - Reclaims disk space by cleaning up dangling images and obsolete containers.
- Automated Installation Scripts - Ships curated shell scripts for the automated installation of cloud CLIs and container runtimes.
- Merged Branch Cleanup - Includes utilities to identify and delete local Git branches that have already been merged into origin.
- Build-Phase Automation - Automates the definition and execution of install and build phases to compile code within cloud environments.
- CLI Abstractions - Simplifies complex CLI commands into short-hand aliases and functions to accelerate operational workflows.
- Code Linters - Executes language-specific linters to ensure code quality across multiple programming dialects.
- Commit History Analysis - Provides shortcuts to visualize commit graphs and compare local repository state against remote branches.
- Shell Environment Configurations - Set up shell tuning and aliases to streamline cloud and container workflows.
- Git API Clients - Retrieves user keys, IP ranges, and repository metadata by performing direct API requests via the shell.
- Git Hook Managers - Automates the installation of pre-commit hooks into local repositories using configuration files.
- Metadata Rewriters - Programmatically corrects author information across the entire repository commit history.
- Installation Scripts - Provides pre-configured scripts for the simplified installation of DevOps tools and build systems.
- Pre-commit Hooks - Integrates shell scripts into git lifecycle events to enforce linting and security rules before commits.
- Cross-Repository Operations - Executes coordinated operations and manages merge requests across multiple GitLab repositories.
- Runtime Version Switching - Allows switching between different binary tool versions based on directory-specific requirements.
- Shell Interaction Optimizations - Enhances shell interaction with custom completion interfaces and case-insensitive path matching.
- CI Workflow Automations - Executes bootstrapping and testing tasks within containerized environments triggered automatically by git commits.
- Batch Resource Operations - Processes multiple cloud accounts or repositories by iterating through lists to execute batch operations.
- Local Pipeline Emulation - Allows executing pipeline configurations on a local machine to debug workflows before deploying them to a remote environment.
- CI/CD Workflows - Offers command-line tools to trigger or cancel GitHub Actions workflow executions across multiple repositories.
- Cloud Billing Alarms - Automates the creation of billing alarms and service accounts for AWS resources.
- Cloud Compute Instance Management - Handles SSH connections and audits service accounts for Google Compute Engine instances.
- Cloud Configuration - Provides automated configuration of project IDs and compute regions within the Cloud SDK.
- Cloud Environment Metadata - Retrieves account IDs and default regions based on the active AWS profile.
- Cloud Asset Inventory Managers - Queries cloud provider APIs to generate consolidated reports of deployed instances and networks.
- Registry Manifest Manipulators - Interact with registries to manipulate image manifests and optimize sizes.
- Image Size Optimization - Minimizes container image footprints by cleaning package caches and validating compatibility.
- Image Auditing - Search registries and list tags by date to verify package versions.
- Runner Provisioning - Provides automation for provisioning GitHub Actions runner instances and managing their registration tokens.
- System Package Manager Installations - Provides scripts to install and upgrade software packages across various operating system distributions.
- Cloud Resource Inventory - Inventories deployed GCP resources and creates service account credentials for automation.
- Cluster Discovery - List available clusters in the current account and region.
- Pipeline Automation - Automates the delivery process by executing build and setup steps within containerized cloud pipelines.
- Platform Project Management - Creates projects, imports repositories, and configures branch protection and mirroring through automated tools.
- Repository Metadata Management - Updates repository descriptions and manages project-level environment variables.
- DNS Record Updaters - Creates and updates DNS records across multiple Cloudflare zones.
- Network Diagnostics - Provides tools to identify gateways and troubleshoot network connectivity.
- Automated SSH Key Management - Automates the addition and deletion of user SSH public keys via API.
- Cloud Auditing Tools - Provides utilities for auditing deployed cloud infrastructure configurations against security best practices.
- Cloud Secret Managers - Includes scripts to update, retrieve, and synchronize secrets from cloud secret management services to clusters.
- Firewall Configurations - Sets up network access rules and firewall configurations to control traffic and security.
- Permission Auditing Tools - Analyzes access control policies to identify risky firewall rules and unused identities.
- CI Repository Secrets - Provides the ability to set environment secrets at repository and organization levels for CI pipelines.
- Security Auditing - Identifies security vulnerabilities in CI/CD pipelines, such as insecure forked pull requests.
- Infrastructure Template Analysis - Scans infrastructure templates for security misconfigurations using static analysis.
- SSH Key Management - Ships tools to retrieve and manage public SSH keys for source control access.
- SSL Certificate Managers - Verifies SSL status and handles the deployment of custom certificates across different zones.
- Environment Variable Managers - Sets and manages variable values at the group and project levels to control application behavior.
- Coding Standards Enforcement - Analyzes source code against configurable rules to detect errors and enforce language style standards.
- Resource State Synchronization - Triggers application synchronization and manages resource whitelists to ensure consistency between the management plane and clusters.
- Infrastructure Metadata - Gathers detailed information on networking and compute resources across all active cloud projects.
- Remote Logging Utilities - Provides tools for accessing and retrieving logs from remote systems over SSH.
- System Statistics Collection - Collects performance metrics and resource usage statistics for system diagnostics.
- Shell Script Development - Extensive collection of DevOps-focused shell scripts and utilities.
