awesome-repositories.com
Blog
MCP
awesome-repositories.com

Discover the best open-source repositories with AI-powered search.

ExploreCurated searchesOpen-source alternativesSelf-hosted softwareBlogSitemap
ProjectMCP serverAboutHow we rankPress
LegalPrivacyTerms
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
tomnomnom avatar

tomnomnom/waybackurls

0
View on GitHub↗

Waybackurls

Waybackurls is a command-line OSINT tool that retrieves every known URL for a given domain from the Wayback Machine archive. It functions as a domain reconnaissance utility, discovering forgotten API endpoints, legacy pages, and hidden files by querying the public web archive API.

The tool processes domains independently and statelessly, reading domain names from standard input and streaming discovered URLs line-by-line to standard output. This design enables seamless integration into Unix command pipelines, allowing users to chain waybackurls with other tools for filtering, sorting, and further analysis. Its concurrent HTTP request pooling accelerates the retrieval of large sets of archived URLs.

Waybackurls supports pattern-based response filtering to extract only relevant URL paths and subdomains, aiding in attack surface mapping and subdomain discovery. The tool collects publicly available historical web data for open-source intelligence and security research, helping identify forgotten endpoints and legacy pages that expand the scope of domain reconnaissance.

AI search

Explore more awesome repositories

Describe what you need in plain English — the AI ranks thousands of curated open-source projects by relevance.

Start searching with AI

Features

  • OSINT Command-Line Utilities - A security utility that retrieves archived web pages and hidden endpoints from public archive APIs.
  • Standard Stream Integration - Reading domain names from standard input and streaming discovered URLs to standard output for chaining with other tools.
  • Historical URL Querying - Retrieves historical URL data by sending structured queries to the Wayback Machine's public CDX API endpoint.
  • Standard Output Streams - Writes discovered URLs one per line to standard output, allowing easy filtering, sorting, and piping to downstream utilities.
  • Attack Surface Mapping - Identifying forgotten endpoints, hidden files, and legacy pages to map a domain's full attack surface.
  • Domain Reconnaissance - Discovers forgotten API endpoints and legacy pages by querying the Wayback Machine archive.
  • Historical URL Retrieval - Fetches all previously indexed paths for a target domain using concurrent request handling.
  • OSINT Data Aggregators - Gathering publicly available historical web data for open-source intelligence and security research.
  • Concurrent URL Processors - Fetching and filtering large sets of archived URLs efficiently using concurrent request handling.
  • STDOUT Data Streams - Outputs discovered URLs line-by-line to standard output for piping into other command-line tools.
  • Subdomain Discovery - Extracting subdomains from archived URL data to expand the scope of domain reconnaissance.
  • URL Pattern Filters - Filters raw archive responses using configurable patterns to extract only relevant URL paths and subdomains.
  • Asset Discovery Tools - Retrieves historical web content from third-party archives.
  • Endpoint Discovery - Fetches URLs from the Wayback Machine.
  • Reconnaissance and Discovery - Fetch all URLs that the Wayback Machine knows about for a domain.
  • Reconnaissance Tools - Fetches historical URLs from the Wayback Machine.
  • URL Discovery and Management - Retrieve historical URLs from the Internet Archive.
  • Web Asset Discovery - Fetches historical URLs from the Wayback Machine.
  • Web Crawling and Scraping - Fetches all known URLs for a domain from the Wayback Machine.
  • Web Security Testing - Extracts historical URLs from the Wayback Machine.
4,601 stars·550 forks·Go·23 views

Star history

Star history chart for tomnomnom/waybackurlsStar history chart for tomnomnom/waybackurls

How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.

Frequently asked questions

What does tomnomnom/waybackurls do?

Waybackurls is a command-line OSINT tool that retrieves every known URL for a given domain from the Wayback Machine archive. It functions as a domain reconnaissance utility, discovering forgotten API endpoints, legacy pages, and hidden files by querying the public web archive API.

What are the main features of tomnomnom/waybackurls?

The main features of tomnomnom/waybackurls are: OSINT Command-Line Utilities, Standard Stream Integration, Historical URL Querying, Standard Output Streams, Attack Surface Mapping, Domain Reconnaissance, Historical URL Retrieval, OSINT Data Aggregators.

Which projects share features with tomnomnom/waybackurls?

Projects with overlapping indexed features include: lc/gau — Gau is a command-line tool and passive URL enumerator designed to discover and aggregate known and historical web… six2dez/reconftw — reconftw is an attack surface management framework and reconnaissance workflow orchestrator designed to automate the… lanmaster53/recon-ng — recon-ng is an open source intelligence reconnaissance framework designed to automate the collection and aggregation… xnl-h4ck3r/waymore — Find way more from the Wayback Machine, Common Crawl, Alien Vault OTX, URLScan, VirusTotal, GhostArchive &… projectdiscovery/katana — Katana is a web crawler and spider designed for security reconnaissance and web application mapping. It functions as a… findomain/findomain — Findomain is a subdomain discovery tool and DNS resolver used for mapping an organization's external attack surface.…

Projects sharing features with Waybackurls

These projects share indexed features with Waybackurls. Shared tags can include platform or build tooling; verify the primary use case before treating a result as a replacement.
  • lc/gaulc avatar

    lc/gau

    5,157View on GitHub↗

    Gau is a command-line tool and passive URL enumerator designed to discover and aggregate known and historical web addresses for specific target domains. It functions as a collection framework that retrieves domain-specific data from public web archives and threat intelligence providers. The tool focuses on passive reconnaissance and open-source intelligence research to map attack surfaces without sending requests directly to target infrastructure. It aggregates data from multiple external sources to identify accessible web endpoints and forgotten pages. The system includes capabilities for r

    Goalienvaultgauhacktoberfest
    View on GitHub↗5,157
  • six2dez/reconftwsix2dez avatar

    six2dez/reconftw

    7,226View on GitHub↗

    reconftw is an attack surface management framework and reconnaissance workflow orchestrator designed to automate the discovery, mapping, and monitoring of external digital assets. It operates as a modular tool-chain pipeline that coordinates a sequence of security tools to perform intelligence gathering and vulnerability scanning. The project distinguishes itself through a cloud-native deployment model that parallelizes scanning workloads across a fleet of remote VPS instances to bypass local resource constraints. It utilizes container-based environment isolation to ensure consistent executio

    Shellbug-bountybugbountybugbounty-tool
    View on GitHub↗7,226
  • lanmaster53/recon-nglanmaster53 avatar

    lanmaster53/recon-ng

    5,698View on GitHub↗

    recon-ng is an open source intelligence reconnaissance framework designed to automate the collection and aggregation of public information. It is a modular intelligence tool that utilizes a system of pluggable modules to harvest target data, resolve DNS queries, and parse web content. The framework is built as an API-driven tool with a programmatic interface to integrate with other security workflows. It is provided as a containerized application, using Docker to ensure a consistent environment for running reconnaissance tasks and managing a persistent data store. Its capabilities cover exte

    Python
    View on GitHub↗5,698
  • projectdiscovery/katanaprojectdiscovery avatar

    projectdiscovery/katana

    17,811View on GitHub↗

    Katana is a web crawler and spider designed for security reconnaissance and web application mapping. It functions as a utility for identifying endpoints, forms, and API structures across web targets by combining standard HTTP request traversal with headless browser automation to render dynamic, JavaScript-heavy content. The tool distinguishes itself through its ability to maintain authenticated sessions and handle complex web interactions, such as automated form submission and captcha resolution. It provides granular control over the discovery process, allowing users to define specific crawl

    Goclicrawlergocrawler
    View on GitHub↗17,811
  • Compare all 30 related projects→