awesome-repositories.com
Blog
MCP
awesome-repositories.com

Descubre los mejores repositorios open-source con nuestra búsqueda potenciada por IA.

ExplorarBúsquedas curadasAlternativas open-sourceSoftware autohospedableBlogMapa del sitio
ProyectoAcerca deCómo clasificamosPrensaServidor MCP
Aviso legalPrivacidadTérminos
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
·
IonicaBizau avatar

IonicaBizau/scrape-it

0
View on GitHub↗
4,074 estrellas·218 forks·JavaScript·MIT·3 vistasionicabizau.net/blog/30-how-to-write-a-web-scraper-in-node-js↗

Scrape It

scrape-it is a Node.js web scraper and HTML parser designed to extract structured data from websites and HTML files. It functions as a web data extraction tool that retrieves specific information from DOM elements and converts web content into usable data fields.

The tool uses CSS selectors to target specific data points and employs schema-driven data mapping to organize unstructured web text into a consistent format. It supports custom value transformation to convert raw extracted strings into specific data formats.

The system provides capabilities for web data extraction and automated content mapping. It can parse HTML content sourced from URLs, raw HTML strings, or local file system storage, and integrates with headless browsers to process content from dynamic web pages.

Features

  • Web Data Extraction - Programmatically scrapes and processes web content to extract specific information using selectors.
  • Schema-Driven Mappings - Maps CSS selectors to predefined output fields to ensure structured and consistent data extraction.
  • Web Element Mappings - Maps HTML elements to structured data fields using CSS selectors to organize web text.
  • CSS Selector - Uses CSS selectors to target and extract specific data points from the document object model.
  • Web Data Extraction Tools - Provides utilities for scraping, parsing, and structuring information from web pages into usable formats.
  • Data Extractions - Retrieves specific text or attributes from HTML documents using CSS selectors and DOM traversal.
  • HTML Parsers - Converts raw HTML strings or local files into structured trees for programmatic data extraction.
  • Web Scrapers - Functions as a Node.js based tool for extracting structured data from websites and HTML files.
  • Web Scraping and Extraction - Parses HTML and uses selectors to extract structured data from target URLs.
  • HTML Parsing - Extracts structured information from HTML files stored locally or generated by browsers.
  • Local File Operations - Processes raw HTML content directly from local file system storage.
  • Value Transformations - Provides custom functions to transform raw extracted strings into specific data formats.
  • Headless Browsers - Integrates with headless browsers to retrieve content from dynamic or complex web pages.
  • HTML String Parsing - Converts raw HTML strings from files or browser outputs into usable data formats.
  • JavaScript Crawling Frameworks - Human-friendly scraper for Node.js.
  • Web Scraping - Human-friendly web scraping library.

Historial de estrellas

Gráfico del historial de estrellas de ionicabizau/scrape-itGráfico del historial de estrellas de ionicabizau/scrape-it

Búsqueda con IA

Explora más repositorios increíbles

Describe lo que necesitas en lenguaje sencillo: la IA clasifica miles de proyectos open-source curados por relevancia.

Start searching with AI

Alternativas open-source a Scrape It

Proyectos open-source similares, clasificados según cuántas características comparten con Scrape It.
  • scrapinghub/portiaAvatar de scrapinghub

    scrapinghub/portia

    9,509Ver en GitHub↗

    Portia is a containerized scraping platform and visual web scraper that enables no-code data extraction. It serves as a Scrapy visual scraping tool and spider generator, allowing users to design and deploy web scrapers through a graphical interface instead of writing manual selector code. The system distinguishes itself by converting visual web page annotations into executable Scrapy spider code and structured JSON specifications. This visual-to-code mapping allows users to define scraping logic and extraction rules through a point-and-click interface, which can then be exported for use in ex

    Python
    Ver en GitHub↗9,509
  • rchipka/node-osmosisR

    rchipka/node-osmosis

    4,110Ver en GitHub↗

    This project is a Node.js web scraping framework designed to automate data extraction through a programmatic workflow of requests, parsing, and document interaction. It functions as a headless web crawler, an HTTP request manager, and a DOM parser and extractor. The framework distinguishes itself by combining a JavaScript execution engine to interact with dynamic content and a hybrid selection system that utilizes both CSS and XPath selectors. It includes specialized middleware for proxy rotation and cookie-jar session management to maintain authenticated states and manage automated traffic.

    JavaScript
    Ver en GitHub↗4,110
  • hickford/mechanicalsoupAvatar de hickford

    hickford/MechanicalSoup

    4,868Ver en GitHub↗

    MechanicalSoup is a Python web automation library designed to simulate browser behavior. It functions as a toolkit for web scraping and automation, providing an HTML parsing engine and an HTTP session manager to interact with websites programmatically. The library enables headless web interaction by mimicking a real user session. It manages persistent state through cookie handling and automatic redirect following, allowing for programmatic website navigation and the simulation of complex browser interactions. Its capabilities cover automated form population and submission using CSS selectors

    Python
    Ver en GitHub↗4,868
  • bda-research/node-crawlerAvatar de bda-research

    bda-research/node-crawler

    6,785Ver en GitHub↗

    node-crawler is a programmable web crawler for Node.js that manages request queues and automates data extraction. It functions as a rate-limited HTTP client and a headless HTML parser, providing the infrastructure to visit large sets of URLs asynchronously while preventing duplicate processing through task deduplication. The project distinguishes itself through a proxy rotation manager that cycles user agents and proxy servers to bypass access restrictions. It utilizes the HTTP/2 protocol to improve request performance and server compatibility during large-scale scraping operations. The syst

    TypeScriptcheeriocrawlerextract-data
    Ver en GitHub↗6,785
Ver las 30 alternativas a Scrape It→

Preguntas frecuentes

¿Qué hace ionicabizau/scrape-it?

scrape-it is a Node.js web scraper and HTML parser designed to extract structured data from websites and HTML files. It functions as a web data extraction tool that retrieves specific information from DOM elements and converts web content into usable data fields.

¿Cuáles son las características principales de ionicabizau/scrape-it?

Las características principales de ionicabizau/scrape-it son: Web Data Extraction, Schema-Driven Mappings, Web Element Mappings, CSS Selector, Web Data Extraction Tools, Data Extractions, HTML Parsers, Web Scrapers.

¿Qué alternativas de código abierto existen para ionicabizau/scrape-it?

Las alternativas de código abierto para ionicabizau/scrape-it incluyen: scrapinghub/portia — Portia is a containerized scraping platform and visual web scraper that enables no-code data extraction. It serves as… rchipka/node-osmosis — This project is a Node.js web scraping framework designed to automate data extraction through a programmatic workflow… hickford/mechanicalsoup — MechanicalSoup is a Python web automation library designed to simulate browser behavior. It functions as a toolkit for… bda-research/node-crawler — node-crawler is a programmable web crawler for Node.js that manages request queues and automates data extraction. It… nanmicoder/crawlertutorial — CrawlerTutorial is a comprehensive Python web scraping tutorial and framework designed for extracting data from static… matthewmueller/x-ray — X-ray is a headless browser web scraper and HTML content crawler designed to extract structured data from websites. It…