awesome-repositories.com
Blog
MCP
awesome-repositories.com

Discover the best open-source repositories with AI-powered search.

ExploreCurated searchesOpen-source alternativesSelf-hosted softwareBlogSitemap
ProjectMCP serverAboutHow we rankPress
LegalPrivacyTerms
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
Gerapy avatar

Gerapy/Gerapy

0
View on GitHub↗
3,505 stars·646 forks·Python·MIT·13 viewsdocs.gerapy.com↗

Gerapy

Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Scrapyd-Client, Scrapyd-API, Django and Vue.js.

Features

  • Web Scraping - Distributed crawler management framework.

Star history

Star history chart for gerapy/gerapyStar history chart for gerapy/gerapy

How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.

AI search

Explore more awesome repositories

Describe what you need in plain English — the AI ranks thousands of curated open-source projects by relevance.

Start searching with AI

Projects sharing features with Gerapy

These projects share indexed features with Gerapy. Shared tags can include platform or build tooling; verify the primary use case before treating a result as a replacement.
  • alir3z4/html2textAlir3z4 avatar

    Alir3z4/html2text

    2,159View on GitHub↗

    Convert HTML to Markdown-formatted text.

    Pythonmarkdownmarkdown-parserpython
    View on GitHub↗2,159
  • alirezamika/autoscraperalirezamika avatar

    alirezamika/autoscraper

    7,297View on GitHub↗

    Autoscraper is an automatic web scraping library and pattern-based data extractor that learns extraction rules from sample data. It identifies and retrieves text, URLs, and HTML elements from web pages by analyzing sample values to replicate data patterns across different URLs. The system functions as a web scraping model manager, allowing users to save and reload learned rules to maintain consistent data extraction. It supports the export and import of scraping rules to a local file system to avoid repeating the training process for the same website. The library covers automated web data ex

    Python
    View on GitHub↗7,297
  • antivanov/js-crawlerantivanov avatar

    antivanov/js-crawler

    257View on GitHub↗

    js-crawler

    TypeScript
    View on GitHub↗257
  • adbar/trafilaturaadbar avatar

    adbar/trafilatura

    5,319View on GitHub↗

    Trafilatura is a Python library and command-line tool for extracting clean, structured text and metadata from web pages. It downloads HTML content, identifies the main body of text, and strips away navigation, ads, and other boilerplate, returning the core article content along with fields like title, author, date, and URL. The tool can also extract user comments and test whether a page contains extractable text, making it a general-purpose web text extraction library. What distinguishes Trafilatura from simpler extractors is its configurable extraction pipeline, which offers high-speed, high

    Pythonarticle-extractorcorpus-buildercorpus-tools
    View on GitHub↗5,319
Compare all 30 related projects→

Frequently asked questions

What does gerapy/gerapy do?

Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Scrapyd-Client, Scrapyd-API, Django and Vue.js.

What are the main features of gerapy/gerapy?

The main features of gerapy/gerapy are: Web Scraping.

Which projects share features with gerapy/gerapy?

Projects with overlapping indexed features include: alir3z4/html2text — Convert HTML to Markdown-formatted text. alirezamika/autoscraper — Autoscraper is an automatic web scraping library and pattern-based data extractor that learns extraction rules from… antivanov/js-crawler — js-crawler. bda-research/node-crawler — node-crawler is a programmable web crawler for Node.js that manages request queues and automates data extraction. It… binux/pyspider — PySpider is a Python web crawling framework designed for automated data extraction. It provides a pipeline for… adbar/trafilatura — Trafilatura is a Python library and command-line tool for extracting clean, structured text and metadata from web…