Consul Template

Consul Template is a configuration renderer and dynamic configuration manager that generates files by populating templates with data from Consul and Vault. It functions as a service discovery template engine and secrets management integrator, transforming cluster catalog and health data into formatted configuration files.

The tool distinguishes itself by acting as a process supervisor and notifier, capable of executing shell commands or restarting applications automatically after templates are updated. It features a long-polling watcher to monitor remote key-value stores and employs a shared locking mechanism to coordinate updates across multiple instances and prevent simultaneous service restarts.

The system covers a broad range of capabilities, including automated secret rotation for PKI certificates and vault credentials, data format transformation for JSON and YAML, and the execution of external binary plugins for custom data processing. It also provides infrastructure bootstrapping and distributed rendering synchronization to reduce API load through leader-based query deduplication.

Features

Configuration File Generation - Generates physical configuration files on disk by populating templates with service discovery and secret data.
Configuration Renderers - Generates configuration files by populating templates with real-time data from Consul and Vault.
Key Retrievals - Retrieves values from a key-value store by path using defaults or existence checks.
Prefix-Based Key Retrievals - Returns all keys and values under a specific prefix from a registry.
Go Template Rendering - Uses Go's text/template engine to dynamically render configuration files by injecting discovery and secret data.
Configuration-Driven Templating Engines - Functions as a template engine that generates configuration files based on cluster catalog and health data.
Dynamic Configuration Management - Generates application configuration files from Consul and Vault that update automatically when values change.
Post-Render Command Executions - Consul Template runs shell scripts or commands automatically after templates render to reload applications or restart services.
Consul Service Discovery - Integrates with Consul for automatic service discovery and health monitoring to drive configuration templates.
Service Discovery Integrations - Provides integration with service registries to dynamically populate configuration files with healthy service instances.
Automated Secret Rotation - Automatically tracks credential expiration and refreshes rotating secrets to ensure continuous service availability.
Secret Management Integrations - Integrates with external secret management services to securely retrieve credentials and certificates for local deployment.
Secret Retrieval Interfaces - Provides mechanisms for securely fetching credentials and sensitive data from protected stores to populate configuration files.
Secrets Management - Provides comprehensive management of secrets from HashiCorp Vault, including versioning and lease renewal.
Secret Vault Integrations - Extracts sensitive credentials from Vault using KV v2 engines and token unwrapping.
Long-Polling Queries - Uses long-polling blocking queries to monitor remote key-value stores for immediate template re-rendering.
Service Discovery Registries - Maintains dynamic registries of service locations, node details, and key-value pairs for configuration rendering.
Service Instance Discovery - Identifies and links to active, healthy service instances via a registry to ensure traffic is routed to available nodes.
Dynamic Configuration Retrieval - Implements real-time retrieval of service information and key-value pairs from a registry to enable dynamic configurations.
Data Format Transformations - Converts data structures into JSON, YAML, TOML, or base64 strings with pretty-printing.
File-Change-Triggered Commands - Executes external shell commands or scripts automatically after a template update to notify or restart services.
Infrastructure Bootstrapping Templates - Renders system files and environment variables based on cluster state during node deployment or startup.
Template Functions - Injects custom functions into the template engine to provide specialized data processing or logic.
Signal-Based Reloading - Triggers application settings refresh by sending OS signals to child processes when configuration files change.
Cluster Node Management - Allows fetching details for all nodes or specific nodes within a cluster catalog for environment-specific configuration.
Distributed Render Synchronization - Uses leader election to render templates once and share the output via a global key-value store.
Distributed Lock Coordination - Prevents simultaneous configuration updates across multiple instances using a shared distributed locking mechanism.
Distributed Locks - Employs distributed locks to synchronize updates across instances and prevent simultaneous service restarts.
Query Deduplication - Elects a single instance to fetch shared data and distribute results to others to reduce API load.
Orchestrator Data Retrieval - Retrieves service discovery data and secrets from a cluster manager to populate configuration templates.
Rendering Pipeline Plugins - Executes external binaries during the rendering pipeline to process data and return results as strings.
Resource Querying - Enables querying of services and variables registered within a cluster orchestrator to use as template data.
Configuration-Driven Process Management - Spawns application processes after rendering and manages their lifecycle via reload or termination signals.
PKI Automation - Automates the retrieval and rotation of PEM-encoded certificates and keys from a PKI engine.
Atomic File Writes - Ensures configuration integrity by writing updates to temporary files before performing an atomic rename.
Standalone Binary Plugin Executions - Executes external binaries to process template data and returns the output as a string.
Process Supervisors - Manages the lifecycle of child processes, automatically restarting or reloading them when configuration templates change.
Configurable File Writing - Saves template content to specific paths with configurable file permissions and ownership.
Process Supervision - Monitors child processes and automatically restarts or reloads them when underlying configuration templates are updated.
Custom Data Processing Plugins - Executes external plugins with optional payloads to perform custom data processing during the rendering pipeline.

twpayne/chezmoi

18,075View on GitHub

chezmoi is a command-line utility designed to manage and synchronize system configuration files across multiple machines. It uses a local Git repository as the single source of truth, allowing users to track, version, and distribute dotfiles while maintaining a consistent state across diverse operating systems and hardware architectures. The project distinguishes itself through a declarative reconciliation model that computes the difference between the current filesystem and the desired state defined in the repository. It features a robust templating engine that processes configuration files

jenkins-x/jx

4,691View on GitHub

jx is a GitOps delivery platform and Kubernetes CI/CD orchestrator designed to automate the building and deployment of applications. It functions as a cloud native pipeline manager that executes container-based build and deployment sequences using a catalog of reusable tasks. The project distinguishes itself through the automated orchestration of preview environments, which are created and destroyed based on pull request activity to enable validation before merging. It employs a GitOps-based state synchronization model to maintain the desired state of clusters by polling git repositories and

alipay/sofa-rpc

3,921View on GitHub

sofa-rpc is a Java-based remote procedure call framework designed for executing remote functions and exchanging data between distributed applications. It serves as a distributed service orchestrator and communication tool, providing a registry-integrated manager to locate and track active network endpoints across a cluster. The framework is built with an interface-based plugin architecture, allowing internal logic and RPC behaviors to be customized through abstraction interfaces. This extensibility enables the implementation of specialized business logic without modifying the core system. Th

fabiolb/fabio

7,332View on GitHub

Fabio is a network gateway that provides reverse proxying, layer 7 traffic management, and automated service discovery mapping. It functions as an HTTP reverse proxy, a gRPC and TCP proxy, and a service discovery gateway to route incoming traffic to healthy backend instances. The project distinguishes itself through deep integration with service registries, specifically acting as a Consul load balancer to automatically synchronize routing tables and update destination targets. It manages diverse traffic types using SNI-based routing for raw TCP streams and maintains full protocol compatibilit

hashicorpconsul-template

Features