# rivermont/spidy

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/rivermont-spidy).**

354 stars · 68 forks · Python · GPL-3.0

## Links

- GitHub: https://github.com/rivermont/spidy
- awesome-repositories: https://awesome-repositories.com/repository/rivermont-spidy.md

## Description

Spidy (/spˈɪdi/) is the simple, easy to use command line web crawler. Given a list of web links, it uses the Python requests library to query the webpages. Spidy then uses lxml to extract all links from the page and adds them to its list. Pretty simple!

## Tags

### Part of an Awesome List

- [Python Crawling Frameworks](https://awesome-repositories.com/f/awesome-lists/devtools/python-crawling-frameworks.md) — Simple command-line web crawler.