allure-framework/allure2

Features

• Execution Data Capture - Collects structured test lifecycle data including steps, timings, attachments, and metadata from multiple frameworks.
• Test Execution Reports - Converts raw test execution data from multiple languages and frameworks into a unified visual HTML report.
• Failure Categorization Rules - Ships a rule engine that groups test failures into user-defined categories for easier analysis.
• Portable HTML Reports - Generates self-contained HTML reports that can be opened directly in a browser without needing a web server.
• Test Performance Profiling - Provides a timeline view to find slow tests and detect parallelization bottlenecks.
• Pipeline Report Publishers - Integrates with continuous integration pipelines to automatically generate and publish test reports.
• CI CD Pipelines - Generate test reports inside a CI/CD pipeline by using the correct tool version and embedding results on the pipeline page.
• Directory Watchers - Provides real-time report regeneration by watching a results directory for new test data.
• Cross-Language Architecture Analysis - Normalizes execution data from different test frameworks into a common intermediate format.
• Behavioral Result Hierarchies - Organizes test results into behavioral hierarchies like epics, features, and stories for BDD-style reporting.
• CI/CD Integrations - Connects with popular CI/CD platforms, IDEs, and multiple testing frameworks to automate report generation.
• Reporting Plugins - Provides a plugin interface for creating custom test reporters that extend the report generation pipeline.
• Result File Collectors - Collects raw test results from multiple sources into a shared directory of JSON files for unified processing.
• Test Status Trend Tracking - Visualize changes in test statuses, transitions between runs, test base size, retries, and categories across historical data.
• Test Result Comparison - Use internal test identifiers to associate results from different test executions for historical trend analysis in reports.
• Aggregate Test Charts - Generates aggregate charts showing test status distribution, severity, and trends in test reports.
• Assertion Step Recording - Records each assertion as a detailed step showing expected and actual values in the report.
• Sub-Step Definition - Splits tests into named sub-steps with parameters for detailed execution reporting.
• Composable Test Steps - Breaks a test into named, nestable sub-steps that log status, parameters, and produce a detailed execution tree.
• Automated Test Report Generators - Processes test results into an HTML report and opens it in a browser via build command or live server.
• Portable Interactive Reports - Processes test data into portable HTML reports featuring visual analytics, trends, and failure categorization.
• Test Suite Formatting - Assigns suite names for hierarchical organization in test reports.
• Story-Based Test Organization - Maps each Gherkin scenario to a user story based on tags, creating a behavior-based hierarchy in the report.
• Pipeline Report Generators - Generates and publishes test reports automatically in continuous integration pipelines with PR summaries.
• Quality Gates - Controls process exit code based on custom rules over test results to enforce pass/fail decisions in CI/CD pipelines.
• Cross-Language Result Aggregation - Aggregates test results from different programming languages and frameworks into a single report.
• Data-Driven Testing - Repeats a scenario outline for each row of an examples table, executing steps with different parameter sets.
• Test Metadata - Enriches test cases with descriptions, links, and metadata for detailed reporting.
• Test Annotations - Adds descriptive labels, tags, severity levels, links, and IDs to tests for richer reporting.
• Runtime Metadata Injectors - Enriches test results at runtime through annotations or API calls without modifying the test framework.
• Test Reporters - Processes and formats test execution results into structured reports.
• Test Result Reporters - Converts raw test execution data into an interactive HTML report for browsing and analysis.
• CI Build Report Access - Integrates test report access directly into CI build pages for viewing and download.
• Context-Enriched Reports - Adds metadata, labels, parameter values, and environment info to tests for comprehensive report details.
• Environment Metadata Embeds - Embeds operating system and runtime version details in the report header for debugging context.
• Environment Metadata Recording - Stores OS and runtime version in the report's overview page.
• Environment Property Embeds - Includes static runtime properties like OS version on the report dashboard for troubleshooting context.
• Hierarchical Test Result Organization - Groups test results into hierarchical trees by behavior, suite, or package for browsing.
• Interactive Report Viewers - Ships an interactive HTML report viewer with filtering, sorting, comparison, and defect categorization built in.
• Live Report Generation - Generates a visual test report during test execution by monitoring a results directory for new data.
• Metadata Enrichment - Enriches test results with custom metadata such as descriptions and links for contextual reporting.
• Self-Contained HTML Exports - Produces a standalone HTML report file that opens directly in a browser without any server dependencies.
• Test Fixture Reporting - Treats setup and teardown functions as separate steps in the test report, with optional display.
• Test Result Labeling - Assigns descriptions, owners, tags, severity levels, identifiers, and parameters to test results for rich reporting.
• Custom Key-Value Labels - Attaches arbitrary key-value metadata to a test result for custom classification not covered by built-in fields.
• Test Severity Labels - Tags a test case with a severity level to prioritize results in the report.
• Test Report Aggregators - Combines test results from independent sources into a single unified report.
• Test Retry Strategies - Automatically re-executes failed tests to mitigate transient failures and improve reliability.
• Hierarchical Step Structuring - Splits test execution into nested sub-steps for detailed failure isolation.
• Local Test History Trackers - Persists test execution history across runs for trend analysis and flaky test detection.
• Package-Based Test Hierarchies - Ships package-based hierarchy classification for organizing test results by source code structure.
• Parameterized Test Reporting - Records parameter names and values with masking options for data-driven test analysis in reports.
• Data-Driven Test Reporting - Reports data-driven test instances with their input parameters displayed alongside each execution result.
• Report Header Metadata - Shows user-defined key-value pairs at the top of the test report to convey build or run context.
• Test Metadata Enrichments - Attaches custom metadata like severity, links, and labels to tests for richer report context.
• Parametrized Test Cases - Runs test logic against multiple input sets using parameterized scenarios.
• Parameter Value Reporting - Reports parameter values alongside parametrized test results for data-driven analysis.
• Report Categorization Labels - Assign short terms and key-value pairs to tests for filtering and organization in the report.
• Test Evidence Management - Embeds screenshots, videos, logs, and HTTP traffic into test results to aid failure diagnosis.
• Test Ownership Assignment - Marks a test result with a named individual responsible for the test's execution or maintenance.
• Test Report Attachments - Embeds screenshots, logs, and other files as visual evidence directly in test reports.
• HTTP Request Captures - Wraps fetch calls to record full request and response details as structured attachments in test reports.
• Parameter and File Attachments - Binds key-value parameters and file attachments to a test case so they appear in the generated report.
• Test Resource Linking - Attaches issue tracker or test management system links to test cases for traceability in the report.
• Test Step Documentation - Records named test steps with parameterized titles for granular execution reporting.
• Test Suite Organization - Assigns suite-based labels to structure test results into parent suite, suite, and sub-suite hierarchy.
• Label-Based Report Hierarchies - Groups test results into behavior-based or suite-based hierarchies for structured reporting.
• Suite Assignment - Assigns suite labels to tests for hierarchical organization in test reports.
• URL Templates - Provides configurable URL templates that transform short identifiers into full external hyperlinks in test reports.
• Self-Contained HTML Reports - Generates self-contained HTML reports that can be opened directly in a browser without a web server.
• Static Report Generation - Generates a self-contained HTML report that can be hosted on static sites for public access.
• Global Test Labeling - Defines labels, including custom ones, via environment variables to categorize tests across the report.
• Environment Variable Label Injection - Apply custom metadata to every test result by reading environment variables that follow a naming convention.
• Flakiness Trend Trackers - Tracks test stability, flakiness, and performance trends over time using historical execution data.
• Automated Report Publishing - Automatically publishes generated test reports to static hosting sites for team access.
• PR Test Result Reporting - Attach a visual test summary with test duration and counts of new, flaky, and retried tests to each pull request.
• Latest Run Summaries - Summarizes the latest test run by displaying statuses, severity, and layer distribution for a quick overview.
• Sensitive Variable Redaction - Removes or masks sensitive fields using string, regex, or custom logic before storing the data in the report.
• Automatic Console Output Capture - Automatically captures stdout, stderr, process errors, and exit codes and includes them in the report.
• Project Health Indicators - Highlight coverage differences, feature-level success rates, environment-specific failures, stability thresholds, and persistence of failed tests.
• Known Issue Mapping - Labels test results as known failures based on a user-defined list.
• Output Capture Utilities - Captures standard output, standard error, and logging during test execution and attaches them to results.
• Test Console Captures - Automatically captures console output and attaches it to test reports for debugging.
• Environment Information Gatherers - Adds static environment properties, such as OS version, to the report for context during debugging.
• Status Change Detection - Identifies tests whose status changed from the previous launch, showing new passes, failures, or broken results.
• Assertion Diff Visualizers - Attaches arbitrary files, byte content, or visual comparison diffs to test results for richer reporting.
• Assertion Instrumentation - Records every Chai assertion as an Allure step by instrumenting assertion prototypes and interfaces.
• Assertion Value Displays - Displays expected and actual values of failed assertions side by side to highlight differences during debugging.
• Gherkin Sub-Step Decomposition - Breaks a single Gherkin step into finer sub-steps using annotations, lambdas, or placeholders for detailed reporting.
• Test Plan Configurations - Reads a test plan file to restrict execution to only the tests listed in it.
• Static Hosting Report Publishers - Generate detailed test reports from a continuous integration pipeline and publish them to a static hosting service, retaining history across runs.
• Custom Quality Gate Rules - Create custom validation rules by defining conditions that test result batches must meet.
• Pull Request - Evaluate quality rules against test results and create a pass/fail status on the pull request.
• Cloud Test History Storage - Persists test execution history in a built-in cloud service for long-term tracking and comparison.
• Configurable Report Hierarchies - Organizes tests into nested groups and steps so the report clearly shows the structure and flow of execution.
• Local Report Serving - Generates a visual test report from execution results and serves it in a browser.
• Flaky Test Analyzers - Automatically marks tests that have both failed and passed within the last five launches, highlighting instability.
• Plugin-Based Output Formats - Uses a plugin system to produce reports in different UI themes, CSV export, Slack notifications, or upload to TestOps.
• Environment Metadata Display - Displays system and configuration details in the test report header for execution context.
• Environment-Specific Report Sections - Groups test results by environment for side-by-side comparison in the report.
• Environment Metadata Attachment - Attaches key-value metadata describing the test environment to the report.
• Static Environment Property Capture - Captures static environment properties (e.g., OS, Java version) into a file that appears on the report’s overview page.
• Custom Rule-Based Categories - Defines custom categories in a JSON file to classify test results based on matching rules for grouped display.
• Setup and Teardown Step Marking - Provides a way to mark setup and teardown methods as visible steps in the test report.
• Test Title and Description Annotations - Add a human-readable title and a formatted description to a test for clarity.
• Test Parameterization - Generates multiple unique test cases by iterating over provided data sets.
• Retry Status Tracking - Marks tests that ran multiple times and received at least two different statuses, indicating potential instability.
• Test Keyword Tagging - Attaches arbitrary text tags to tests for filtering and categorization in the report.
• Structured HTTP Exchange Attachments - Records each HTTP request and response with timestamps, headers, body, and error details as a JSON attachment.
• HTTP Traffic Recording - Captures live HTTP request/response pairs and attaches them to test reports with automatic sensitive data redaction.
• Test Severity Classifications - Marks tests with severity levels to highlight the impact of failures in reports.
• Test Owner Assignments - Mark tests with a priority level and a responsible team member to help prioritize failure investigations.
• Test Case Annotations - Annotates a function as a test case, automatically initializing the report context and managing lifecycle.
• Parameterized Test Name Formatting - Formats test titles using parameter values by passing them to a formatting string in the report.
• Custom Test Runners - Exposes programmatic hooks to start, stop, and annotate test cases in custom test runners.
• Test Labeling Systems - Labels tests with arbitrary key-value metadata for categorization and filtering in reports.
• Concurrent Reporters - Allows running additional reporters alongside the primary one in the same test run.
• Execution Timeline Views - Displays test execution events along a chronological timeline to track pass/fail timing.
• Plan-Based Test Selection - Filters test execution to only run tests listed in an external test plan file.
• Global Artifact Attachments - Attaches global artifacts to test reports not tied to individual tests.
• Test Status Categorization - Displays test results with Passed, Failed, Skipped, and Broken statuses using distinct colors for filtering.
• Custom Failure Category Rules - Groups failed and broken test results into user-defined categories based on error messages and labels.
• Failure Classification Rules - Map exceptions to Failed or Broken statuses and use custom logic to distinguish assertion errors from environment issues.
• Method-Level Step Decomposition - Splits a test method into named sub-steps so the report displays execution flow at a finer granularity.
• Test Lifecycle Hooks - Exposes hooks to modify test results before or after standard processing in the reporting pipeline.
• Test Resource Linking - Attaches URLs under link, issue, or task-management icons to connect tests to related items.
• Per-Type Link Template Patterns - Construct full URLs from short identifiers by applying a configurable template for each link type in test reports.
• Test Report Link Templates - Creates URL templates that turn short identifiers into full links for test artifacts or references.
• Per-Type Link Templates - Builds full URLs from short identifiers using a template pattern to link test cases to external resources.
• Structured HTTP Exchange Attachments - Records each HTTP exchange as a structured JSON attachment, including timestamps, request, response, and error data.
• Coding Quality Rules - Check test results against a set of rules and fail the run when rules are violated, blocking deployment.

Istoric stele