# postmodern/spidr

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/postmodern-spidr).**

837 stars · 107 forks · Ruby · MIT

## Links

- GitHub: https://github.com/postmodern/spidr
- awesome-repositories: https://awesome-repositories.com/repository/postmodern-spidr.md

## Description

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

## Tags

### Part of an Awesome List

- [Web Crawling](https://awesome-repositories.com/f/awesome-lists/data/web-crawling.md) — Versatile library for spidering domains and links.
- [Test Automation Frameworks](https://awesome-repositories.com/f/awesome-lists/devtools/test-automation-frameworks.md) — Web spidering library for Ruby.