projectdiscovery/httpx
Httpx
httpx is a suite of tools and libraries for HTTP reconnaissance, infrastructure discovery, and DNS resolution. It functions as a command line toolkit for extracting metadata and status codes from HTTP targets and CIDR ranges, as well as a Go library for integrating these probing capabilities into custom programs.
The project distinguishes itself through specialized infrastructure profiling, using TLS fingerprinting to extract JARM hashes and certificate details. It identifies underlying components such as CDN usage, Autonomous System Numbers, and CNAMEs to map web server software and infrastructure profiles.
The toolkit covers broad capability areas including automated web probing, DNS resolution via DoH, TCP, and UDP, and security posture analysis of certificates and headers. It also manages HTTP authentication testing using Bearer tokens and BasicAuth, and employs retry logic and backoff strategies to navigate network obstacles.
The project supports execution through containerized images for consistent deployment across operating systems.
Features
- Web Asset Probing - Performs large-scale HTTP requests across CIDR ranges and host lists to identify active web services.
- Infrastructure Profiling - Extracts TLS certificates and JARM hashes to create detailed infrastructure profiles of target servers.
- Command-Line HTTP Clients - Provides a fast command line tool for extracting metadata and status codes from HTTP targets and CIDR ranges.
- Infrastructure Discovery Automators - Scans network environments to identify CDN usage, ASN details, CNAMEs, and web server software.
- Infrastructure Discovery Tools - Maps web servers and assets by probing hosts and URLs to identify status codes, page titles, and headers.
- Resolution Testing - Translates domain names into IP addresses using custom resolvers over DoH, TCP, or UDP for testing purposes.
- Resolution Utilities - Provides a tool for translating domain names into IP addresses using DoH, TCP, or UDP protocols.
- Attack Surface Management - Analyzes TLS certificates, JARM hashes, and security headers to evaluate the external security posture of a network.
- TLS Fingerprinting - Extracts JARM hashes and analyzes TLS handshakes to identify server software and infrastructure profiles.
- Custom DNS Resolvers - Provides capabilities to translate domain names into IP addresses using custom resolvers over DoH, TCP, or UDP.
- Multi-Protocol Resolvers - Translates domain names via DoH, TCP, or UDP protocols to resolve targets before initiating HTTP requests.
- Service Probing - Extracts metadata such as status codes, page titles, and server headers from hosts, URLs, and CIDR ranges.
- Authentication Configurations - Applies domain-specific credentials and headers based on external configuration files during the request lifecycle.
- Integration Libraries - Provides a Go library that allows embedding probing logic and runners into custom external programs.
- HTTP Client Libraries - Ships as a Go package for integrating multi-purpose HTTP probing and metadata extraction into custom programs.
- HTTP Request Customization - Allows customizing HTTP requests with BasicAuth, Bearer tokens, and custom headers to access protected resources.
- HTTP Authentication Schemes - Verifies access to protected web resources at scale using BasicAuth, Bearer tokens, and custom cookies.
- Security Configuration Auditing - Inspects TLS certificates and Content Security Policy headers to evaluate the security configuration of web servers.
- Concurrent Task Runners - Uses a worker-based model to execute multiple network requests in parallel for accelerated scanning.
- Pipeline Component Modularization - Implements a modular pipeline to sequentially extract status codes, page titles, and server headers from targets.
- Retry and Backoff Logic - Wraps the HTTP client with automated retry and backoff logic to navigate unstable network connections and firewalls.
- HTTP Probing and Crawling - Fast and multi-purpose HTTP toolkit for probing and analysis.
- Reconnaissance and Discovery - Fast and multi-purpose HTTP toolkit for probing web servers.
- Reconnaissance Tools - Fast, multi-purpose HTTP toolkit for probing and analysis.
- Technology Fingerprinting - Fast HTTP toolkit for reliable probing and analysis.
- Directory Bruteforcing - Multi-purpose HTTP toolkit for probing and verifying web hosts.
- Web Security Testing - Probes and discovers HTTP-based services with various checks.
