1 repository
Awesome GitHub RepositoriesCredentialed Web Crawling
Capabilities for a web crawler to authenticate against restricted sites using various credential schemes.
Distinct from Basic Authentication: Focuses on the crawler's ability to use credentials to access protected content, rather than the implementation of the authentication server itself.
Explore 1 awesome GitHub repository matching security & cryptography · Credentialed Web Crawling. Refine with filters or upvote what's useful.
Awesome Credentialed Web Crawling GitHub Repositories
Găsește cele mai bune repo-uri cu AI.Vom căuta cele mai potrivite repository-uri folosind AI.
- digininja/cewl
digininja/CeWL
2,575Vezi pe GitHubCeWL is a custom wordlist generator and web crawling security tool designed to extract unique words and metadata from websites. It functions as an OSINT metadata extractor and security scanner, identifying potential passwords and usernames by analyzing HTML and JavaScript content. The tool differentiates itself by combining recursive spidering with metadata extraction, allowing it to collect email addresses, author names, and creator metadata from web pages and linked files. It also captures domains, subdomains, and path components to include in generated lists. Broad capabilities include we
Accesses restricted websites using Basic or Digest authentication credentials to reach private content.
Ruby
