Buf

Buf is a toolchain for managing the full lifecycle of Protocol Buffers schemas. It provides a set of tools for schema governance, including linting to enforce style guides, a breaking change detector to ensure backward compatibility, and a system for producing language-specific source code via local or remote plugins.

The project distinguishes itself through a remote schema registry that centralizes the hosting, versioning, and distribution of Protocol Buffers modules. This registry allows for server-side enforcement of governance policies, such as blocking updates that introduce backward-incompatible changes, and serves pre-built language SDKs to consumers.

The toolchain covers a broad surface of capabilities including module dependency management with graph-based resolution and locking, canonical AST-based formatting, and developer tooling via a Language Server Protocol for real-time editor integration. It also includes utilities for API testing, such as converting binary wire formats to JSON and invoking remote services.

The toolchain can be deployed as a containerized image for portable pipelines or hosted as an on-premises registry within a private network.

Features

Schema-Driven Code Generators - Produces language-specific types and handlers using versioned configurations and plugins.

Schema Registries - Provides a centralized registry for hosting, versioning, and distributing Protocol Buffers modules and generated SDKs.

Protocol Buffers - Provides a central repository for managing the lifecycle and distribution of Protocol Buffer schemas.

API Compatibility Analysis - Compares different versions of a compiled schema to detect breaking changes in the binary encoding.

Wire-Format Compatibility Management - Detects breaking changes based on whether the consumer interacts via binary wire format or source code.

Pre-built SDK Distributions - Hosts pre-compiled client libraries to allow consumers to install SDKs without local code generation.

Schema Registry Governance Policies - Validates every pushed schema against breaking-change and uniqueness rules at the registry level.

Protobuf Code Generators - Automatically generates type-safe source code and client stubs from Protocol Buffer definitions using local or remote plugins.

Remote Plugin Execution - The project executes generator binaries on a remote server to produce source code without installing toolchain binaries locally.

Schema Management Toolchains - Provides a complete toolchain for managing Protocol Buffers schemas from initial formatting and linting to SDK generation.

Module Management - Provides systems for importing and managing modular packages of Protocol Buffers schemas.

Server-Side Hook Enforcement - Runs lint and policy checks server-side to reject pushes that violate organizational standards.

Code Generators - Produces source code in target languages by running plugins over schemas and injecting automated file options.

Schema Module Publishing - Hosts versioned bundles of schema files and their dependencies as addressable modules for distribution.

Canonical Formatters - Parses schema files into an abstract syntax tree to rewrite them into a consistent and sorted style.

API Schema Linting - Validates schema files against style and structure rules to ensure consistency and quality.

API Specification Governance - Enforces style guides and detects breaking changes in Protocol Buffers to ensure long-term backward compatibility.

Breaking Change Detectors - Compares a current schema against a previous version to identify modifications that would break clients or servers.

Schema - Compares schema versions to identify modifications that would break binary wire-format or source-code compatibility.

Dependency Graph Resolution - Tracks and fetches versioned schema dependencies from a remote registry to ensure reproducible builds.

Schema Backward Compatibility Guards - Blocks updates to schema files that would introduce backward-incompatible changes.

Schema Compatibility Lifecycle Checks - Executes breaking change detection locally, in CI, or on the server to catch issues before production.

Schema Policy Enforcement - Validates schemas against linting and compatibility rules on the server before allowing modules to be published.

Schema Linters - Validates schema files against style and structural rules to ensure consistency and quality.

Schema Language Servers - Implements a background process that provides real-time autocomplete and navigation for schemas within text editors.

Binary Descriptor Sets - Compiles source files into binary descriptor sets to distribute validated schemas without requiring original source files.

Schema Change Approvals - Routes failed updates to owners for manual approval before publishing changes.

Custom Lint Rule Plugins - Provides a plugin interface to extend the linter with user-defined rules via external binaries.

Custom Compatibility Rule Definitions - Allows teams to enable or disable specific compatibility rules to enforce organization-specific governance policies.

IDE Language Plugins - Provides a language server to enable real-time autocomplete, navigation, and validation for schemas in editors.

Language Server Protocol Implementations - Implements the Language Server Protocol to provide editor intelligence and formatting for Protocol Buffer schemas.

Linting Configurations - Applies predefined rule sets to enforce progressively stricter coding conventions for schemas.

Linting Plugins - Integrates external binaries as plugins to enforce project-specific linting requirements.

Module Organization - Groups directories of schema files into versionable units for building, linting, and publishing.

Plugin Management - Publishes and manages custom check plugins to extend linting and compatibility validation.

Registry Module Exports - Downloads schema files from a remote registry into a local directory.

Schema Formatters - Rewrites schema files to follow a canonical style for consistent formatting across the codebase.

Validated Artifact Visibility - Hides pending or rejected commits from SDK registries to ensure consumers receive validated schemas.

Version Pinning - Enforces specific versions of generated libraries using labels or commit IDs to ensure reproducible builds.

Binary Descriptor Distributions - Compiles source files into binary descriptor sets to distribute validated schemas without requiring original source files.

CI Formatting Checks - Returns a non-zero exit code when formatting would change files to enforce style in pipelines.

Abstract Syntax Tree Parsing - Parses schema files into an abstract syntax tree to rewrite them into a consistent and sorted style.

Compatibility Layers - Implements architectural patterns to ensure schema updates maintain support for legacy clients and older API versions.

Governance Policy Management - Groups check rules into named configurations that apply to specific organizations.

Language-Agnostic APIs - Separates package and namespace options from source files into a separate config to keep schemas neutral.

Language-Neutral Schema Configuration - Moves language-specific package and namespace settings into a configuration file to keep sources language-neutral.

CI Linting Enforcement - Runs lint rules on every pull request via a CI/CD job to catch issues before merging.

Remote Plugin Execution Runtimes - Executes plugins on a remote registry to produce source code locally without requiring a local toolchain.

Schema Configuration Decoupling - Separates package and namespace options from source files into a separate config to keep schemas neutral.

Schema Namespace Enforcement - Prevents duplicate file paths and type names across modules to ensure namespace consistency.

Workspace Module Coordination - Defines a common configuration for multiple modules, including shared linting rules.

API Testing - Provides tools for invoking remote services and converting binary wire formats to JSON for API validation.

Schema Compilation Verification - Builds the workspace to ensure all schema files are syntactically correct.

Serialization Libraries - Modern tool for managing and building protocol buffers.

bufbuildbuf

Features

Star history