kubernetes/kops
Kops
kops is a Kubernetes cluster provisioner and lifecycle manager designed to automate the creation, maintenance, and destruction of production-grade clusters on cloud infrastructure. It functions as a declarative infrastructure manager, synchronizing the live state of a cluster with versioned manifests stored in remote object storage to ensure idempotent operations.
The project distinguishes itself by offering comprehensive automation for the entire cluster lifecycle, including high-availability control plane deployment, incremental rolling updates, and automated version upgrades. It also serves as an infrastructure-as-code exporter, capable of generating Terraform configurations from the current state of a deployed cluster.
Beyond provisioning, it covers a broad operational surface including automated node and pod scaling, etcd data store management, and complex networking configurations such as dual-stack IPv6 and CNI integration. It also manages identity and security through OIDC authentication integration, cloud IAM role mapping, and x509 certificate lifecycle management.
The tool provides a command-line interface with support for shell autocompletion.
Features
- Cluster Lifecycle Management - Provides comprehensive automation for the installation, configuration, and version upgrades of production-grade Kubernetes clusters.
- Kubernetes Cluster Provisioning - Automates the creation, configuration, and deployment of production-grade Kubernetes clusters on cloud infrastructure.
- Monitoring and Health - Validates the current cluster configuration and health to ensure the live state matches the declarative specification.
- Provider Abstractions - Provides a translation layer that maps high-level resource definitions to specific cloud provider API requests.
- Cloud Infrastructure Management - Provisions and manages virtual machines, load balancers, and networks to support container orchestration environments.
- Cloud Infrastructure Scaling - Interacts with cloud APIs to programmatically adjust compute resource capacity and optimize container performance.
- Cluster Configuration Management - Updates cluster settings and instance group configurations to modify the desired state of infrastructure.
- Cluster Definitions - Defines new cluster configurations using specification files or command-line arguments for registration.
- Idempotent Operations - Ensures that cluster maintenance, upgrades, and destruction operations are idempotent via a state-sync model.
- Infrastructure State Updates - Modifies the configuration and state of existing clusters to apply infrastructure changes.
- Cluster Lifecycle Managers - Uses version-controlled manifests to define infrastructure, providing an audit trail for cluster configuration changes.
- Cluster Management - Orchestrates and maintains the underlying cloud resources, virtual machines, and networks that support the cluster.
- Resource-Driven Node Scaling - Automatically adjusts the number of cluster nodes based on resource demand to optimize capacity.
- Cluster Upgrades - Manages infrastructure-level version updates for production clusters while maintaining system availability.
- CNI Plugins - Deploys networking providers and CNI plugins to manage pod and node connectivity.
- Elastic Load Balancers - Provisions and manages cloud-native Elastic Load Balancers and integrates them with security services.
- Control Plane Deployment Tools - Deploys redundant control-plane nodes to eliminate single points of failure and ensure cluster availability.
- State Synchronization - Uses versioned manifests and templates to ensure idempotent updates and consistent cluster state synchronization.
- Distributed State Store Deployments - Provisions persistent cloud volumes and configures DNS synchronization to establish a resilient state store.
- High Availability Conversions - Transitions a cluster from a single-node control plane to a high-availability multi-master configuration.
- High Availability Deployments - Distributes control-plane nodes across multiple availability zones to maintain operational continuity during outages.
- Node Pool Scaling - Allows adjusting the capacity of specific instance groups by synchronizing cloud state with desired configuration.
- Infrastructure State Synchronization - Continuously aligns the live infrastructure state with versioned manifests to ensure configuration consistency.
- Cilium - Installs and configures Cilium as the primary eBPF-powered networking interface for the cluster.
- Kubernetes Cluster Management - Provides tools for provisioning, configuring, and maintaining the ongoing health and lifecycle of Kubernetes clusters.
- Release Upgrades - Manages Kubernetes version upgrades through manual configuration or automated stable-channel updates.
- Version Upgrades - Executes Kubernetes cluster version updates and rolling node replacements to maintain current releases.
- Production-Grade Provisioning - Configures VPCs, state store buckets, and dedicated API server nodes to ensure production-ready networking and control planes.
- State Store Persistence - Persists cluster configuration and metadata in durable remote object storage to maintain a single source of truth.
- Pod Autoscaling - Automatically adjusts the number of active pod replicas based on CPU and memory utilization metrics.
- Load-Balanced Cluster Access - Secures cluster entry points by routing traffic through DNS and load balancers using SSL certificates.
- Manifest-Based State Reconciliation - Reconciles live infrastructure against versioned manifests stored in remote object storage for idempotent cluster management.
- Rolling Update Orchestrators - Orchestrates sequential node replacement and workload draining to apply updates without causing service outages.
- Automated DNS Managers - Automates the creation and resolution of DNS records by associating clusters with hosted zones.
- CNI Implementations - Deploys Calico as the container networking interface to manage pod-to-pod communication.
- DNS-Based Discovery - Sets up DNS records for cluster discovery using public, private, or gossip-based resolution.
- Cloud VPC Provisioning - Deploys clusters into existing VPCs or creates new virtual private networks using specified CIDR blocks.
- Kubernetes VPC Deployments - Deploys Kubernetes clusters into existing virtual private clouds by reusing existing network IDs.
- Cluster Node Bootstrappers - Automates the installation of the operating system, container runtime, and orchestration agents for new cluster nodes.
- API Access Control - Controls API server accessibility by switching load balancers between internet-facing and internal-only modes.
- Certificate Lifecycle Management - Automates the issuance, renewal, and rotation of x509 certificates to secure internal and external cluster communication.
- Trust Revocations - Marks specific keypairs as distrusted to prevent their use for authentication within the cluster.
- Cluster Identity Trust Establishment - Establishes trust relationships for cryptographic keypairs to secure identity and communication within the cluster.
- Configuration Manifests - Uses structured YAML manifests to define the desired state of the cluster and its infrastructure components.
- Declarative Resource Updates - Updates the desired state of cluster resources using declarative YAML manifests or standard input.
- Rolling Node Updates - Performs sequential replacement of cluster nodes to apply updates without causing service outages.
- Cluster Backups - Performs periodic snapshots of the cluster data store to remote object storage for disaster recovery.
- Cluster Restorations - Recovers the cluster data store from historical snapshots stored in object storage to restore system state.
- Cluster State Validations - Verifies that the live cluster state and configuration match the intended specification to ensure health.
- Configuration File Generators - Produces YAML configuration files by injecting variables from files or command-line sets into templates.
- Machine Specification Updates - Modifies the instance type, boot image, and root volume settings for groups of nodes.
- Dry Run Simulations - Simulates infrastructure changes before application to ensure safe, idempotent updates.
- Infrastructure Configuration Templates - Generates cloud resource configurations by injecting variables into predefined YAML infrastructure templates.
- Cache Server Deployments - Deploys a caching layer to reduce the request load on the API server in large-scale clusters.
- Cloud-Init Configurations - Injects custom user data and shell scripts during instance launch to configure the operating system and runtime.
- State-to-Terraform Exporters - Generates Terraform configurations from the current cluster state to allow management via external code.
- Cloud Network Security Groups - Integrates clusters with pre-defined cloud security groups to enforce organizational network security policies.
- Cloud Resource Deletions - Automates the teardown of all cloud resources, including instance groups and secrets, associated with a cluster.
- Addons Management - Installs and configures modular software extensions and components to add specialized functionality to the cluster.
- Custom Addon Deployments - Installs user-defined resources from a remote manifest store using a polling mechanism.
- Node Metadata Labeling - Assigns custom metadata labels to nodes to control pod scheduling and workload placement.
- VPC CNI Implementations - Provides cloud-managed IP addresses directly to pods using the VPC CNI model for native cloud network integration.
- CNI Provider Configurations - Simplifies the selection and deployment of popular container networking providers during cluster setup.
- CNI Provider Migrations - Orchestrates the process of switching between different container network interface (CNI) implementations across cluster nodes.
- Compute Resource Orchestration - Manages compute capacity by creating, editing, and performing rolling updates on instance groups.
- Cluster Configuration Templates - Creates standardized cluster configurations using templates to ensure consistency across environments.
- Scheduling Controls - Applies taints and labels to nodes to precisely control pod scheduling and affinity.
- Dual-Stack Networking - Provides comprehensive support for IPv6-only pods and dual-stack networking including CIDR allocation and routing.
- Node Runtime Hooks - Injects containers, hooks, and files directly into cluster nodes via a centralized manifest to customize the host environment.
- Node Draining - Implements node draining by monitoring cloud events to migrate workloads before instance termination.
- Hardware-Based Instance Selection - Automatically selects the most appropriate cloud instance types based on required CPU and memory specifications.
- Control Plane Capacity Scaling - Increases control plane capacity by adding dedicated instance groups specifically for the API server.
- Hybrid Instance Management - Manages a strategic mix of on-demand and spot instances within node groups to optimize cost and availability.
- Cluster Component Addons - Deploys and manages common functional extensions and networking providers with minimal configuration.
- Pre-initialized Instance Pools - Maintains a pool of pre-initialized instances to significantly accelerate cluster scaling operations.
- Infrastructure State Exporters - Generates Terraform files that represent the currently provisioned cloud resources.
- Instance Launch Configurations - Defines boot and configuration parameters, including instance whitelists and spot percentages, for node deployment.
- Instance Placement Policies - Implements server group affinity policies to distribute virtual machines across availability zones for high availability.
- Instance Type Specification - Defines specific hardware requirements for CPU and memory to automate the selection of eligible cloud instances.
- Autoscaler Parameter Tuning - Tunes autoscaler behavior by configuring cooldown periods and evaluation thresholds to regulate cluster growth.
- Cluster Destructions - Tears down all cloud resources provisioned for a cluster to stop billing and infrastructure usage.
- Load Balancer Health Management - Maintains traffic stability during upgrades by blocking the removal of unhealthy nodes from load balancer targets.
- Node Association - Automatically links compute instance groups to external load balancer target groups for seamless traffic distribution.
- Multi-Architecture Deployments - Supports deploying clusters across diverse hardware architectures, including ARM64 and x86 nodes.
- Node Boot Hooks - Executes custom scripts or containers on nodes during the boot process to perform system setup and driver installation.
- Node Group Provisioning - Provisions groups of virtual machines based on specified machine types, scaling limits, and availability zones.
- Multi-Zone Distribution - Provisions groups of virtual machines across multiple availability zones to ensure high availability and fault tolerance.
- Cluster State Synchronization - Implements a dedicated etcd cluster to synchronize agent state across large clusters.
- Private Cluster Deployments - Provisions clusters isolated from the public internet using internal IP addresses and secure bastion hosts.
- Spot Instance Orchestration - Manages the lifecycle of interruptible spot instances to optimize cloud compute costs.
- Managed Instance Groups - Defines and manages cloud instance groups with control over machine types, scaling limits, and spot instance usage.
- Mixed Purchasing Models - Optimizes costs by combining on-demand and spot instances within a single node group.
- Declarative State Triggers - Triggers infrastructure updates automatically when the desired state in a YAML configuration file is modified.
- CIDR Management - Manages network CIDR allocations and defines node port ranges for cluster-wide networking.
- Cloud Native Networking - Assigns cloud network interfaces and native subnet IPs to pods for direct cloud connectivity.
- Kubernetes Node Failure Detections - Runs a daemon on every node to detect and report hardware or software issues as cluster events.
- Gateway API Integrations - Integrates with the Gateway API to provide an extensible method for configuring ingress traffic.
- Kubernetes Network Orchestration - Configures container networking interfaces, dual-stack IPv6, and isolated private topologies within cloud environments.
- OS Customization - Customizes the operating system during the boot process using cloud-init user data and package installation.
- Network Policy Enforcement - Implements and manages pod-to-pod connectivity and network policy enforcement within the cluster.
- Cloud Topology Provisioning - Defines whether nodes are launched in internet-accessible or isolated private subnets to control ingress.
- Proxy Configurations - Routes cluster and infrastructure traffic through a forward proxy for restricted network connectivity.
- Egress Routing Configurations - Configures egress routing for private subnets using NAT gateways or transit gateways.
- Route Table Customization - Modifies subnet route tables to define specific traffic paths toward instances or gateways.
- Proxy Implementations - Replaces standard NodePort implementations with BPF to improve network performance and efficiency.
- Network Dataplanes - Implements a packet forwarding layer using eBPF to replace the standard dataplane and proxy.
- Cloud Instance Image Selections - Allows specifying the operating system image for node groups using IDs, aliases, or parameters.
- Access Restrictions - Restricts access to node SSH and master HTTPS endpoints to specific allowed CIDR blocks.
- Boot Volume Encryption - Encrypts instance boot volumes using cloud-provider managed encryption keys.
- Certificate Authorities - Signs cluster certificates using a user-provided CA keypair to maintain control over the trust chain.
- OIDC Integrations - Integrates with OpenID Connect providers to authenticate users via issuer URLs and client identifiers.
- Pod - Automatically injects required authentication tokens and environment variables into pods for cloud identity integration.
- Kubernetes Identity Integration - Integrates OIDC authentication and cloud IAM roles to manage permissions for users and service accounts.
- Instance-Level IAM Assignments - Assigns cloud identity permissions to all pods on a node via instance profiles.
- OIDC Identity Hosting - Hosts OIDC discovery documents to allow Kubernetes service accounts to assume cloud IAM roles.
- Secure SSH Access - Manages secure shell access to nodes by configuring public keys and restricting source IP addresses.
- Agent-to-Agent Encryptions - Secures communication between cluster agents using transparent IPsec or WireGuard encryption.
- Pod-to-Pod Encryptions - Integrates WireGuard to provide automatic, transparent encryption for all data transmitted between pods.
- Identity Mapping - Maps cloud-based identities to cluster users and groups to control access through external authentication providers.
- Configuration Versioning - Exports cluster specifications to version-controlled files to maintain a historical audit trail of changes.
- Specification Reconciliation - Reconciles live cloud infrastructure with revised specifications, including an optional preview mode for changes.
- Cluster State Retrieval - Retrieves and filters the current operational state of cluster resources for inspection.
- Node Identity Policies - Extends cloud identity policies for cluster nodes and bastion hosts to grant necessary access to cloud services.
- IP Access Restrictions - Restricts API server and SSH access to specific IP addresses or CIDR ranges to secure the cluster perimeter.
- OS Patch Management Policies - Defines policies to determine if operating system security updates are applied automatically or externally.
- Resource Metrics - Collects granular container resource metrics to drive built-in autoscaling pipelines.
- Cluster Provisioning - Handles production-grade Kubernetes installation and management.
