Open-source alternatives to Hakrawler

Waybackurls is a command-line OSINT tool that retrieves every known URL for a given domain from the Wayback Machine archive. It functions as a domain reconnaissance utility, discovering forgotten API endpoints, legacy pages, and hidden files by querying the public web archive API. The tool processes domains independently and statelessly, reading domain names from standard input and streaming discovered URLs line-by-line to standard output. This design enables seamless integration into Unix command pipelines, allowing users to chain waybackurls with other tools for filtering, sorting, and furt

Naabu is a port scanner library and tool that probes hosts for open ports using SYN, CONNECT, and UDP methods to identify active services. It functions as a Go library for embedding port scanning into programs, and as a standalone tool that accepts targets as hostnames, IP addresses, CIDR ranges, or ASN numbers. The tool discovers live hosts before scanning, filters ports by range or top lists, and can integrate with Nmap for service version detection. The project distinguishes itself through its SYN-based port probing approach that sends TCP SYN packets and analyzes responses without complet

CeWL is a custom wordlist generator and web crawling security tool designed to extract unique words and metadata from websites. It functions as an OSINT metadata extractor and security scanner, identifying potential passwords and usernames by analyzing HTML and JavaScript content. The tool differentiates itself by combining recursive spidering with metadata extraction, allowing it to collect email addresses, author names, and creator metadata from web pages and linked files. It also captures domains, subdomains, and path components to include in generated lists. Broad capabilities include we

Crawler4j is a multi-threaded Java web crawler and spider designed for high-volume web traversal and content extraction. It functions as a polite crawling framework that enables the discovery and indexing of HTML and binary content across multiple websites. The project distinguishes itself through a persistent crawling model that serializes session state to local storage, allowing the engine to resume indexing after a crash or interruption. It includes a politeness controller to regulate request frequency and delays, preventing server overloading and IP blocking. The system covers a broad ra

Crawlee-python is a web crawling framework for building scalable scrapers using Python. It serves as a comprehensive tool for web scraping automation, providing a system to extract structured data from websites using both lightweight HTTP requests and headless browser automation. The framework is distinguished by its anti-bot evasion capabilities, which include browser fingerprint impersonation and tiered proxy rotation to bypass detection systems and solve challenges such as Cloudflare. It also incorporates artificial intelligence for autonomous website navigation and schema-based data extra

Feroxbuster is an HTTP directory brute forcer and web resource enumerator designed to discover hidden files and directories on web servers. It functions as a recursive URL scanner that identifies unlinked endpoints and API resources by combining wordlist-based scanning with automated crawling. The tool operates as a proxy-aware fuzzer, allowing network requests to be routed through HTTP or SOCKS proxies for traffic interception or anonymity. It utilizes recursive directory crawling to automatically queue discovered paths and find nested content. The system includes capabilities for discovery

RustScan is a high-speed TCP network scanner written in Rust, designed for security reconnaissance and network mapping. It functions as an automated port discovery engine that identifies open ports on remote hosts using IPv6 addresses, CIDR ranges, or bulk input files. The tool is built for rapid surface area discovery, utilizing parallel port processing and OS-aware performance optimizations to identify active services. It allows for scan precision tuning through adjustable connection timeout thresholds and concurrent request controls to balance speed and accuracy. The system integrates wit

LinkFinder is a security reconnaissance and static analysis tool designed for JavaScript endpoint discovery. It extracts absolute and relative URLs and parameters from JavaScript files to map the attack surface of web applications and identify hidden API routes. The tool operates through static code analysis and regular expression pattern matching to find endpoints without executing the source code. It includes a data processor for importing exported files from Burp Suite, enabling the batch analysis of multiple JavaScript assets in a single execution. The system provides capabilities for do

Masscan is a command-line network scanner designed for large-scale discovery and infrastructure reconnaissance. It identifies open ports across specific network segments or the entire internet by probing vast address ranges with high efficiency. The tool functions as an asynchronous packet engine, bypassing standard operating system kernel networking stacks to transmit raw packets directly from application memory. The project distinguishes itself through a specialized architecture that manages millions of concurrent connections by separating packet transmission and reception into independent

Quickly discover exposed hosts on the internet using multiple search engines.

A command-line utility designed to discover URLs for a given domain in a simple, efficient way. It works by gathering information from a variety of passive sources, meaning it doesn't interact directly with the target but instead gathers data that is already publicly available.

Find way more from the Wayback Machine, Common Crawl, Alien Vault OTX, URLScan, VirusTotal, GhostArchive & Intelligence X!

HostHunter a recon tool for discovering hostnames using OSINT techniques.

Amass is a network attack surface mapper and reconnaissance framework designed to discover and map the external, internet-facing infrastructure of a target organization. It functions as an open source intelligence tool that identifies public network boundaries and locates hidden or forgotten subdomains to define an organization's total reachable footprint. The project utilizes passive-source data aggregation from external APIs and public databases alongside active DNS brute-forcing and recursive subdomain expansion. It employs a graph-based asset mapping system to visualize the relationships

The fastest dork scanner written in Go.

httpx is a suite of tools and libraries for HTTP reconnaissance, infrastructure discovery, and DNS resolution. It functions as a command line toolkit for extracting metadata and status codes from HTTP targets and CIDR ranges, as well as a Go library for integrating these probing capabilities into custom programs. The project distinguishes itself through specialized infrastructure profiling, using TLS fingerprinting to extract JARM hashes and certificate details. It identifies underlying components such as CDN usage, Autonomous System Numbers, and CNAMEs to map web server software and infrastr

Take a list of domains, crawl urls and scan for endpoints, secrets, api keys, file extensions, tokens and more

Gobuster is a command-line security utility designed for brute-force discovery of hidden infrastructure and content. It operates by systematically testing wordlists against target network services to identify files, directories, subdomains, and cloud storage buckets. The tool utilizes a concurrent worker pool to execute these requests in parallel, ensuring efficient scanning across various network environments. The project distinguishes itself through a modular plugin architecture that supports multiple discovery modes, including HTTP, DNS, and TFTP. This design allows for protocol-agnostic r

Gau is a command-line tool and passive URL enumerator designed to discover and aggregate known and historical web addresses for specific target domains. It functions as a collection framework that retrieves domain-specific data from public web archives and threat intelligence providers. The tool focuses on passive reconnaissance and open-source intelligence research to map attack surfaces without sending requests directly to target infrastructure. It aggregates data from multiple external sources to identify accessible web endpoints and forgotten pages. The system includes capabilities for r

dirsearch is a command-line security tool and web path scanner used for discovering hidden directories and files on web servers. It functions as a recursive directory fuzzer and brute-force utility that identifies undocumented paths and sensitive files using wordlists and HTTP status codes. The tool distinguishes itself through template-driven path generation and an automated HTTP response filter that uses status codes, content length, and regex patterns to isolate valid targets. It supports recursive directory crawling to map complex web structures and provides state-persistence serializatio

Seatbelt is a C# offensive security framework and host security auditor designed to perform endpoint surveys on Windows systems. It functions as a modular tool for identifying vulnerabilities, misconfigurations, and security-relevant artifacts on both local and remote hosts. The project distinguishes itself through a module-based check system that allows for the integration of custom security command units. It features a security event log parser to track logon and process activity, alongside a credential extraction utility for gathering browser history, saved passwords, and cloud credentials

Anubis is a command-line security reconnaissance framework designed for subdomain enumeration and attack surface mapping. It functions as a utility for security professionals to identify, catalog, and visualize the external digital footprint of an organization by discovering all subdomains associated with a target domain. The tool distinguishes itself through a modular resolver pipeline that integrates passive reconnaissance from third-party security APIs and public certificate transparency logs. It combines this data with active discovery methods, including recursive DNS brute-forcing and al

This project is a library of pre-defined keyboard scripts and payloads designed for execution via HID injection hardware. It provides a collection of scripted keystroke sequences specifically for the Flipper Zero and other compatible hardware to automate tasks on target computers. The library includes HID attack scripts and USB Rubber Ducky payloads that automate commands to bypass manual user interaction. These scripts are designed to perform system reconnaissance, exfiltrate data, and manipulate user interfaces. The available payloads cover several capability areas, including automated sys

Iron is a Rust web framework used for building concurrent web applications and APIs. It functions as a concurrent HTTP server and provides a stateless route dispatcher to map incoming URL paths and globs to specific handler functions. The project is centered around a middleware-based request pipeline, which allows the request-response cycle to be extended via plugins and modifiers. It utilizes a thread-safe state container to store shared application memory accessible across all concurrent handlers and middleware. The framework covers broad capability areas including dynamic API routing, sta

PrivescCheck is a PowerShell-based security auditing tool designed to scan Windows configurations for potential privilege escalation paths and local host vulnerabilities. It functions as a vulnerability assessment utility that analyzes system settings and registry configurations to identify weaknesses that could allow unauthorized administrative access. The tool automates internal security reconnaissance and post-exploitation data collection to gather environmental information. It serves as a security compliance reporter by exporting scan results into structured formats, including HTML, CSV,

Puma is a concurrent HTTP server for Ruby applications that implements the Rack interface. It operates as a clustered web server, using a combination of worker processes and threads to handle multiple simultaneous web connections via TCP ports or UNIX domain sockets. The server features a master-worker process model that utilizes multiple CPU cores and employs copy-on-write preloading to reduce memory usage. It supports zero-downtime restarts through socket-handover capabilities, allowing application updates without dropping pending network requests. The project includes a token-authenticate