TorBot est une suite d'outils spécialisés pour accéder et explorer les services cachés sur le réseau Tor. Il fonctionne comme un crawler onion et un web scraper qui découvre des services cachés en naviguant sur les domaines .onion pour extraire les titres de pages, les noms d'hôtes et les descriptions de sites.
Les fonctionnalités principales de dedsecinside/torbot sont : TOR and Anonymity, Recursive Web Discovery, Onion Crawlers, Proxy-Enabled Scrapers, SOCKS5 Proxies, Web Crawling, Network Link Mappers, Network Relationship Visualizers.
Les alternatives open-source à dedsecinside/torbot incluent : remitchell/python-scraping — This project is a Python web scraping library and automated data collection suite. It provides tools for extracting… rchipka/node-osmosis — This project is a Node.js web scraping framework designed to automate data extraction through a programmatic workflow… s-rah/onionscan — OnionScan is a free and open source tool for investigating the Dark Web. six2dez/reconftw — reconftw is an attack surface management framework and reconnaissance workflow orchestrator designed to automate the… megadose/onionsearch — OnionSearch is a script that scrapes urls on different .onion search engines. awesome-selfhosted/awesome-selfhosted — This project is a community-curated directory of open-source software designed for deployment in private server…
This project is a Python web scraping library and automated data collection suite. It provides tools for extracting structured data from websites, implementing web crawlers to navigate site links, and parsing HTML DOM structures to isolate specific elements and attributes. The toolkit includes a pipeline for processing unstructured text and cleaning raw web content to extract meaningful information. It also features capabilities for image data extraction and the integration of external APIs to retrieve structured data from remote endpoints. The system covers broad capability areas including
This project is a Node.js web scraping framework designed to automate data extraction through a programmatic workflow of requests, parsing, and document interaction. It functions as a headless web crawler, an HTTP request manager, and a DOM parser and extractor. The framework distinguishes itself by combining a JavaScript execution engine to interact with dynamic content and a hybrid selection system that utilizes both CSS and XPath selectors. It includes specialized middleware for proxy rotation and cookie-jar session management to maintain authenticated states and manage automated traffic.
OnionScan is a free and open source tool for investigating the Dark Web.
reconftw is an attack surface management framework and reconnaissance workflow orchestrator designed to automate the discovery, mapping, and monitoring of external digital assets. It operates as a modular tool-chain pipeline that coordinates a sequence of security tools to perform intelligence gathering and vulnerability scanning. The project distinguishes itself through a cloud-native deployment model that parallelizes scanning workloads across a fleet of remote VPS instances to bypass local resource constraints. It utilizes container-based environment isolation to ensure consistent executio