# s0rg/crawley

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/s0rg-crawley).**

341 stars · 18 forks · Go · MIT

## Links

- GitHub: https://github.com/s0rg/crawley
- awesome-repositories: https://awesome-repositories.com/repository/s0rg-crawley.md

## Description

The unix-way web crawler

## Tags

### Part of an Awesome List

- [Content Discovery](https://awesome-repositories.com/f/awesome-lists/devtools/content-discovery.md) — Unix-style web scraper and crawler.
- [General Utilities](https://awesome-repositories.com/f/awesome-lists/devtools/general-utilities.md) — Unix-way web crawler.