awesome-repositories.com
© 2026 Bringes Technology SRL·VAT RO45896025·hello@bringes.io
MCPSitemapPrivacyTerms
Web Scraping Frameworks · Awesome GitHub Repositories

2 repos

Awesome GitHub RepositoriesWeb Scraping Frameworks

Comprehensive toolkits and engines designed to extract structured data from websites by defining navigation rules or using language models.

Explore 2 awesome GitHub repositories matching web development · Web Scraping Frameworks. Refine with filters or upvote what's useful.

  1. Home
  2. Web Development
  3. Web Automation and Scraping
  4. Web Scraping and Automation
  5. Web Scraping Frameworks

Awesome Web Scraping Frameworks GitHub Repositories

Describe the repository you're looking for…
We'll search the best matching repositories with AI.
  • unclecode/crawl4ai

    unclecode/crawl4ai

    60,452GitHubView on GitHub↗

    Crawl4AI is an AI-powered web crawling and data extraction engine designed to transform complex web content into structured formats. It functions as a headless browser orchestrator, enabling the navigation of dynamic websites, the execution of custom scripts, and the capture of visual assets like screenshots and PDFs.

    Python
  • scrapy/scrapy

    scrapy/scrapy

    59,824GitHubView on GitHub↗

    Scrapy is a comprehensive framework designed for automated web data extraction and large-scale crawling. It operates on an asynchronous, event-driven engine that manages non-blocking network requests and data processing tasks, allowing for the efficient retrieval of structured information from web documents using path-

    Pythoncrawlercrawlingframework

Explore sub-tags

  • AI-Powered Web CrawlersCrawlers that utilize language models to interpret and structure complex web content.
  • Web ScrapersAutomated systems for navigating websites and extracting structured data.