awesome-repositories.com
Blog
MCP
awesome-repositories.com

Discover the best open-source repositories with AI-powered search.

ExploreCurated searchesOpen-source alternativesSelf-hosted softwareBlogSitemap
ProjectMCP serverAboutHow we rankPress
LegalPrivacyTerms
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
zhuyingda avatar

zhuyingda/webster

0
View on GitHub↗
559 stars·52 forks·JavaScript·GPL-3.0·11 views

Webster

Webster is a reliable web crawling and scraping framework written with Node.js, used to crawl websites and extract structured data from their pages.

Features

  • JavaScript Crawling Frameworks - Framework for scraping AJAX and JavaScript-rendered content.
  • Web Scraping - Crawler framework for Ajax and JS-rendered content.

Star history

Star history chart for zhuyingda/websterStar history chart for zhuyingda/webster

How this analysis was created: This summary and feature list are AI-generated from collected project material and can contain mistakes. Stars, license and language are imported from GitHub. Inclusion does not mean that we have tested or audited this project. Check the source documentation for any feature you depend on. Learn more on our About page.

AI search

Explore more awesome repositories

Describe what you need in plain English — the AI ranks thousands of curated open-source projects by relevance.

Start searching with AI

Projects sharing features with Webster

These projects share indexed features with Webster. Shared tags can include platform or build tooling; verify the primary use case before treating a result as a replacement.
  • bda-research/node-crawlerbda-research avatar

    bda-research/node-crawler

    6,785View on GitHub↗

    node-crawler is a programmable web crawler for Node.js that manages request queues and automates data extraction. It functions as a rate-limited HTTP client and a headless HTML parser, providing the infrastructure to visit large sets of URLs asynchronously while preventing duplicate processing through task deduplication. The project distinguishes itself through a proxy rotation manager that cycles user agents and proxy servers to bypass access restrictions. It utilizes the HTTP/2 protocol to improve request performance and server compatibility during large-scale scraping operations. The syst

    TypeScriptcheeriocrawlerextract-data
    View on GitHub↗6,785
  • brendonboshell/supercrawlerbrendonboshell avatar

    brendonboshell/supercrawler

    381View on GitHub↗

    Supercrawler is a Node.js web crawler. It is designed to be highly configurable and easy to use.

    JavaScript
    View on GitHub↗381
  • antivanov/js-crawlerantivanov avatar

    antivanov/js-crawler

    257View on GitHub↗

    js-crawler

    TypeScript
    View on GitHub↗257
  • cgiffard/node-simplecrawlercgiffard avatar

    cgiffard/node-simplecrawler

    2,133View on GitHub↗

    simplecrawler is designed to provide a basic, flexible and robust API for crawling websites. It was written to archive, analyse, and search some very large websites and has happily chewed through hundreds of thousands of pages and written tens of gigabytes to disk without issue.

    JavaScript
    View on GitHub↗2,133
Compare all 30 related projects→

Frequently asked questions

What does zhuyingda/webster do?

Webster is a reliable web crawling and scraping framework written with Node.js, used to crawl websites and extract structured data from their pages.

What are the main features of zhuyingda/webster?

The main features of zhuyingda/webster are: JavaScript Crawling Frameworks, Web Scraping.

Which projects share features with zhuyingda/webster?

Projects with overlapping indexed features include: cgiffard/node-simplecrawler — simplecrawler is designed to provide a basic, flexible and robust API for crawling websites. It was written to… ionicabizau/scrape-it — scrape-it is a Node.js web scraper and HTML parser designed to extract structured data from websites and HTML files.… antivanov/js-crawler — js-crawler. brendonboshell/supercrawler — Supercrawler is a Node.js web crawler. It is designed to be highly configurable and easy to use. bda-research/node-crawler — node-crawler is a programmable web crawler for Node.js that manages request queues and automates data extraction. It… lapwinglabs/x-ray — X-Ray is a web scraping framework and asynchronous web crawler designed to extract structured data from websites. It…