1 repo
Awesome GitHub RepositoriesWeb Scraping Techniques
Methods for navigating and extracting data from websites with access restrictions.
Distinguishing note: Focuses on bypassing access blocks during data collection, distinct from general archiving.
Explore 1 awesome GitHub repository matching development tools & productivity · Web Scraping Techniques. Refine with filters or upvote what's useful.
Awesome Web Scraping Techniques GitHub Repositories
Find the best repos with AI.We'll search the best matching repositories with AI.
- ArchiveBox/ArchiveBox
ArchiveBox/ArchiveBox
26,876View on GitHubArchiveBox is a self-hosted archiving tool designed for personal digital preservation and research data management. It functions as an automated web preservation engine that monitors URL inputs from bookmarks, browser history, or manual entries to capture and store permanent, offline copies of web content. By utilizing headless browser automation, the system renders dynamic web pages to ensure that captured snapshots, PDFs, and media assets remain accurate and accessible even if the original source disappears. The project distinguishes itself through a modular extractor pipeline and a task-qu
Working Around Sites that Block Archiving — a named example documented in this learning resource.
Pythonarchiveboxbackupsbookmark-archiver
