# brendonboshell/supercrawler

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/brendonboshell-supercrawler).**

0 stars · 0 forks

## Links

- GitHub: https://github.com/brendonboshell/supercrawler
- awesome-repositories: https://awesome-repositories.com/repository/brendonboshell-supercrawler.md

## Tags

### Part of an Awesome List

- [JavaScript Crawling Frameworks](https://awesome-repositories.com/f/awesome-lists/devtools/javascript-crawling-frameworks.md) — Crawler with custom handlers and rate limiting.
- [Web Scraping](https://awesome-repositories.com/f/awesome-lists/devtools/web-scraping.md) — Crawler with support for robots.txt and rate limiting.