HypothesisWorks/hypothesis

Features

• Deterministic Random Data Generation - Generates reproducible random inputs based on defined strategies to uncover edge cases through property-based testing.
• Example Shrinking - Simplifies complex failing inputs by iteratively reducing data size until the smallest possible counterexample is found.
• Generative Testing - Provides a property-based testing framework that generates random inputs to validate code invariants and uncover edge cases.
• Primitive Generators - Generates basic scalar data types such as integers, floats, and booleans for testing.
• Backend Abstractions - Implements a backend-agnostic architecture that decouples high-level data strategies from the underlying random value generation engine.
• Deferred Evaluation - Handles deeply nested data using lazy definitions that resolve strategies only during the generation process.
• Structured Random Data Generation - Creates randomized lists, tuples, and class instances by composing multiple data generation rules.
• Test Data Generators - Creates realistic and randomized data instances, including complex structures and database models, for software testing.
• Arbitrary Data Generation - Generates random instances of types to verify system invariants using property-based testing.
• State Transition Testing - Executes sequences of interdependent operations to verify that a system maintains correct behavior across state transitions.
• Schema-Driven Generators - Automatically maps type annotations and database schemas to corresponding data generation strategies during test setup.
• State-Transition Execution - Executes randomized sequences of interdependent operations to verify system invariants across state transitions.
• Collection Generation - Produces randomized lists, sets, tuples, and dictionaries with configurable sizes and uniqueness constraints.
• Composite Generation Strategies - Defines complex strategies by drawing values from other strategies within a decorated function.
• Constraint-Based Data Generation - Combines primitive generators into complex strategies to produce randomized inputs based on specified type constraints.
• Coverage-Guided Fuzzing Integration - Converts raw bytestrings from a coverage-guided fuzzer into structured test cases to leverage data generation and shrinking.
• Failing Case Shrinkers - Simplifies complex failing inputs into the smallest possible example to make debugging more efficient.
• Counterexample Identification - Detects and reports the exact counterexample that causes a test to fail for precise debugging of edge cases.
• Fuzzing Integration Layers - Converts raw byte buffers from coverage-guided fuzzers into structured test cases with shrinking support.
• Python Libraries - Provides a comprehensive property-based testing framework specifically implemented for Python.
• State Machine Modeling - Defines structured state machines with rules that execute in random order to verify complex system properties.
• State Machine Testing - Verifies system behavior by executing sequences of interdependent operations in a randomized order.
• Test Case Shrinking - Implements algorithms to simplify complex failing test inputs into the smallest possible reproducible examples.
• Text and String Generation - Produces strings and characters based on custom alphabets, Unicode categories, codecs, or regular expressions.
• Dynamic Input Generation - Mixes data drawing and test assertions within the same function to generate inputs dynamically based on intermediate results.
• Database Testing - Implements a specialized storage mechanism to track and manage the history of test cases and failures.
• Fuzzing And Analysis - Combines coverage-guided fuzzing with structured data generation to uncover vulnerabilities and crashes.
• Strategy Combinators - Combines multiple data generation rules to create complex inputs such as lists of mixed types.
• Binary Serialization Formats - Uses binary serialization formats to store failing test inputs for consistent reproduction across different environments.
• Custom Generator Definitions - Defines new input generators by combining existing strategies with custom code to produce complex data shapes.
• Django Model Generation - Produces randomized data instances that respect Django field validators and database constraints.
• Model and Form Generation - Produces randomized instances of database models and forms by inferring strategies from field types.
• Generation Engine Swapping - Changes the engine for primitive type generation to use methods like coverage-guided fuzzing or SMT solvers.
• Recursive Data Generation - Produces deeply nested data structures by combining a base strategy with deferred definitions.
• Strategy Inference - Creates data generation strategies by analyzing definitions like JSON Schema, GraphQL, or model definitions.
• Type-to-Strategy Registries - Associates custom types with specific strategies for automatic lookup during the data generation process.
• Strategy - Applies a transformation function to every value produced by a strategy to change its format or structure.
• Strategy Type Annotations - Uses generic type annotations to specify the expected output type of data generation strategies.
• Custom Generation Provider Registration - Connects custom types to specific generation strategies using entry points for automatic recognition during testing.
• Test Runner Plugins - Connects to the Pytest runner to provide command-line control over seeds, verbosity, and generation statistics.
• Test Example Synchronization - Shares and replays failing test inputs across different machines or pipelines using a networked database.
• Randomized Instance Generation - Produces instances of classes or callables by drawing arguments from strategies or type annotations.
• Input Provider Backends - Allows the creation of custom providers that control the distribution of generated inputs by overriding primitive type methods.
• Crash Reproduction - Uses serialized binary blobs from failure reports to recreate specific failing test cases in different environments.
• DataFrame Generation - Produces tabular data structures by defining column shapes, data types, and row generation strategies.
• Dependent Data Generation - Generates complex data structures where one value depends on another by drawing multiple values within a single strategy.
• External Fuzzer Integrations - Integrates with coverage-guided fuzzers by converting raw byte buffers into structured test cases.
• Test Execution Overrides - Controls the number of generated examples, replay behavior, and verbosity for individual tests or entire suites.
• Test Failure Debugging Tools - Re-runs tests using serialized binary blobs or fixed seeds to recreate previously observed failures.
• Non-Deterministic Failure Detection - Identifies tests that fail on an initial run but pass when the same input is replayed.
• Test Runners - Combines property-based data generation with traditional test fixtures by working alongside standard testing tools.
• Test Data Management - Stores and retrieves generated values through named collections to flow data between operations during stateful runs.
• Failed Test Rerunning - Stores failing inputs in a database to automatically replay them in subsequent test runs.
• Generation Rule Composition - Combines multiple data sources to produce a random selection from a set of different types or constants.
• Grammar-Based Generation - Produces strings that conform to a specified context-free grammar using EBNF specifications.
• Guided Input Generation - Uses numeric observation metrics to steer the search toward inputs likely to falsify a property.
• Numerical Array Generation - Produces arrays with specified data types, shapes, and element strategies for mathematical computations.
• Parallel Test Execution - Executes test suites across multiple processes or threads to increase performance while maintaining thread safety.
• Automated Test Code Generation - Creates property-based test source code by analyzing type annotations and signatures to verify patterns like idempotency.
• Generated Input Filtering - Excludes invalid or unwanted test cases using predefined predicates to ensure inputs meet specific criteria.
• Framework Integrations - Combines property-based testing with traditional parameterized test suites by working alongside standard test runners and fixtures.
• Strategy-Level Filtering - Excludes specific values from a strategy based on a predicate to prevent invalid data from entering a test.
• Temporal Data Generation - Produces randomized dates, times, datetimes, timedeltas, and timezone identifiers for testing.
• Test Example Databases - Defines a custom storage backend to save, fetch, and delete failing test examples across environments.
• Generated Example Filtering - Excludes generated examples that do not meet specific criteria using filters to ensure only valid data reaches the test logic.
• Testing Frameworks - Enables property-based testing for finding edge cases.
• Testing Tools - Listed in the “Testing Tools” section of the Awesome Python awesome list.

Star history