schemathesis/schemathesis
Schemathesis
Schemathesis is a property-based testing tool and fuzzer for schema-based APIs. It analyzes OpenAPI and JSON Schema specifications to automatically generate test cases that identify crashes, schema violations, and validation bypasses. The project functions as a contract validator and security scanner, verifying that a live server strictly adheres to its defined specifications.
The framework distinguishes itself through stateful API testing, which chains multiple related requests together to uncover bugs that only emerge during complex, multi-step user workflows. It also utilizes response-driven fuzzing to adapt its testing strategy based on server responses, allowing for deeper exploration of API state transitions.
The tool covers a broad range of capabilities including automated API regression, property-based test generation, and minimal test case reduction. It provides utilities to export failures as minimal reproduction commands and supports reporting through network archive files and the Allure visual format.
Schemathesis operates as a command line runner that integrates into CI/CD pipelines, with support for file-based configuration of authentication headers and reporting settings.
Features
- API Fuzzers - Automatically generates test cases from OpenAPI schemas to identify crashes, schema violations, and validation bypasses.
- Fuzz Testing - Provides automated fuzz testing that uses schema-driven semi-random inputs to uncover edge cases and API crashes.
- API Contract Validation - Verifies that a live server strictly adheres to its OpenAPI or JSON Schema specification across all possible inputs.
- Request Chaining - Maintains a sequence of dependent API calls to simulate complex user workflows and identify state-related bugs.
- REST API Auditing - Automatically discovers API vulnerabilities by auditing REST endpoints and exploiting parameters and headers via schema-based fuzzing.
- Stateful - Tests sequences of related API calls to find bugs that only appear during multi-step user workflows.
- Stateful Workflow Testing - Chains multiple operations into sequences to uncover bugs that emerge during realistic multi-step user journeys.
- Property-Based Testing - Implements a property-based testing methodology to verify API invariants using automatically generated random inputs.
- REST API Fuzzers - Sends semi-random and edge-case data to API endpoints based on schemas to find crashes and validation bugs.
- Schema-Based Input Generators - Creates test data by mapping schema types to generators that produce random and boundary-value inputs.
- Stateful Testing - Chains multiple related API requests together to uncover bugs that only emerge during complex multi-step user workflows.
- Test Case Generators - Analyzes API definitions to automatically generate input data and edge-case scenarios for comprehensive coverage.
- Schema-Driven Test Generation - Analyzes API specifications to create test inputs that identify crashes, schema violations, and validation bypasses.
- Dynamic Schema Parsing - Analyzes OpenAPI and JSON Schema specifications at runtime to determine the structure of target endpoints.
- API CI/CD Integrations - Provides a command line runner for automating API contract validation and stability testing within CI/CD pipelines.
- Response-Driven Fuzzing - Adjusts the generation strategy based on server responses to explore deeper API state transitions.
- CI/CD Pipeline Integrations - Operates as a command line runner that integrates into CI/CD pipelines to automate API quality checks.
- Multi-Phase Execution Strategies - Executes a coordinated sequence of verification strategies including random fuzzing and stateful request chaining.
- Bug Reproduction Scripting - Generates minimal curl commands from discovered failures to help developers recreate and fix API bugs.
- Test Result Reporters - Produces visual reports and machine-readable files containing failure steps and reproduction commands for external analysis.
- Automated Regression Verification - Integrates schema-based test generation into CI/CD pipelines to catch regressions and breaking changes in every build.
- Test Case Minimization - Simplifies complex failing API request sequences into the smallest possible set of steps to reproduce a bug.
- API Testing Tools - A tool for property-based testing of API schemas.
- GraphQL Development Tools - Runs arbitrary queries to find server errors.
- Testing and Debugging - Property-based testing tool for OpenAPI-defined applications.
- 测试框架 - Property-based testing library for API schemas.
- Testing Tools - Listed in the “Testing Tools” section of the Awesome Python awesome list.
