awesome-repositories.com
Blog
awesome-repositories.com

Descoperă cele mai bune repository-uri open source cu căutare AI.

ExploreazăCăutări recomandateAlternative open-sourceSoftware self-hostedBlogHartă site
ProiectDespreCum realizăm clasamentulPresăServer MCP
LegalConfidențialitateTermeni
© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com
·
IonicaBizau avatar

IonicaBizau/scrape-it

0
View on GitHub↗
4,074 stele·218 fork-uri·JavaScript·MIT·3 vizualizăriionicabizau.net/blog/30-how-to-write-a-web-scraper-in-node-js↗

Scrape It

scrape-it este un scraper web și parser HTML pentru Node.js, conceput pentru a extrage date structurate de pe site-uri web și fișiere HTML. Funcționează ca un instrument de extracție a datelor web care preia informații specifice din elementele DOM și convertește conținutul web în câmpuri de date utilizabile.

Instrumentul folosește selectori CSS pentru a viza puncte de date specifice și utilizează maparea datelor bazată pe schemă pentru a organiza textul web nestructurat într-un format consistent. Suportă transformarea personalizată a valorilor pentru a converti șirurile brute extrase în formate de date specifice.

Sistemul oferă capabilități pentru extracția datelor web și maparea automată a conținutului. Poate analiza conținut HTML provenit din URL-uri, șiruri HTML brute sau stocare locală și se integrează cu browsere headless pentru a procesa conținutul din pagini web dinamice.

Features

  • Web Data Extraction - Programmatically scrapes and processes web content to extract specific information using selectors.
  • Schema-Driven Mappings - Maps CSS selectors to predefined output fields to ensure structured and consistent data extraction.
  • Web Element Mappings - Maps HTML elements to structured data fields using CSS selectors to organize web text.
  • CSS Selector - Uses CSS selectors to target and extract specific data points from the document object model.
  • Web Data Extraction Tools - Provides utilities for scraping, parsing, and structuring information from web pages into usable formats.
  • Data Extractions - Retrieves specific text or attributes from HTML documents using CSS selectors and DOM traversal.
  • HTML Parsers - Converts raw HTML strings or local files into structured trees for programmatic data extraction.
  • Web Scrapers - Functions as a Node.js based tool for extracting structured data from websites and HTML files.
  • Web Scraping and Extraction - Parses HTML and uses selectors to extract structured data from target URLs.
  • HTML Parsing - Extracts structured information from HTML files stored locally or generated by browsers.
  • Local File Operations - Processes raw HTML content directly from local file system storage.
  • Value Transformations - Provides custom functions to transform raw extracted strings into specific data formats.
  • Headless Browsers - Integrates with headless browsers to retrieve content from dynamic or complex web pages.
  • HTML String Parsing - Converts raw HTML strings from files or browser outputs into usable data formats.
  • JavaScript Crawling Frameworks - Human-friendly scraper for Node.js.
  • Web Scraping - Human-friendly web scraping library.

Istoric stele

Graficul istoricului de stele pentru ionicabizau/scrape-itGraficul istoricului de stele pentru ionicabizau/scrape-it

Căutare AI

Explorează mai multe repository-uri excelente

Descrie ce ai nevoie în limbaj simplu — AI-ul sortează mii de proiecte open source selectate în funcție de relevanță.

Start searching with AI

Întrebări frecvente

Ce face ionicabizau/scrape-it?

scrape-it este un scraper web și parser HTML pentru Node.js, conceput pentru a extrage date structurate de pe site-uri web și fișiere HTML. Funcționează ca un instrument de extracție a datelor web care preia informații specifice din elementele DOM și convertește conținutul web în câmpuri de date utilizabile.

Care sunt principalele funcționalități ale ionicabizau/scrape-it?

Principalele funcționalități ale ionicabizau/scrape-it sunt: Web Data Extraction, Schema-Driven Mappings, Web Element Mappings, CSS Selector, Web Data Extraction Tools, Data Extractions, HTML Parsers, Web Scrapers.

Care sunt câteva alternative open-source pentru ionicabizau/scrape-it?

Alternativele open-source pentru ionicabizau/scrape-it includ: scrapinghub/portia — Portia is a containerized scraping platform and visual web scraper that enables no-code data extraction. It serves as… rchipka/node-osmosis — This project is a Node.js web scraping framework designed to automate data extraction through a programmatic workflow… hickford/mechanicalsoup — MechanicalSoup is a Python web automation library designed to simulate browser behavior. It functions as a toolkit for… bda-research/node-crawler — node-crawler is a programmable web crawler for Node.js that manages request queues and automates data extraction. It… nanmicoder/crawlertutorial — CrawlerTutorial is a comprehensive Python web scraping tutorial and framework designed for extracting data from static… matthewmueller/x-ray — X-ray is a headless browser web scraper and HTML content crawler designed to extract structured data from websites. It…

Alternative open-source pentru Scrape It

Proiecte open-source similare, clasificate după numărul de funcționalități comune cu Scrape It.
  • scrapinghub/portiaAvatar scrapinghub

    scrapinghub/portia

    9,509Vezi pe GitHub↗

    Portia is a containerized scraping platform and visual web scraper that enables no-code data extraction. It serves as a Scrapy visual scraping tool and spider generator, allowing users to design and deploy web scrapers through a graphical interface instead of writing manual selector code. The system distinguishes itself by converting visual web page annotations into executable Scrapy spider code and structured JSON specifications. This visual-to-code mapping allows users to define scraping logic and extraction rules through a point-and-click interface, which can then be exported for use in ex

    Python
    Vezi pe GitHub↗9,509
  • rchipka/node-osmosisR

    rchipka/node-osmosis

    4,110Vezi pe GitHub↗

    This project is a Node.js web scraping framework designed to automate data extraction through a programmatic workflow of requests, parsing, and document interaction. It functions as a headless web crawler, an HTTP request manager, and a DOM parser and extractor. The framework distinguishes itself by combining a JavaScript execution engine to interact with dynamic content and a hybrid selection system that utilizes both CSS and XPath selectors. It includes specialized middleware for proxy rotation and cookie-jar session management to maintain authenticated states and manage automated traffic.

    JavaScript
    Vezi pe GitHub↗4,110
  • hickford/mechanicalsoupAvatar hickford

    hickford/MechanicalSoup

    4,868Vezi pe GitHub↗

    MechanicalSoup is a Python web automation library designed to simulate browser behavior. It functions as a toolkit for web scraping and automation, providing an HTML parsing engine and an HTTP session manager to interact with websites programmatically. The library enables headless web interaction by mimicking a real user session. It manages persistent state through cookie handling and automatic redirect following, allowing for programmatic website navigation and the simulation of complex browser interactions. Its capabilities cover automated form population and submission using CSS selectors

    Python
    Vezi pe GitHub↗4,868
  • bda-research/node-crawlerAvatar bda-research

    bda-research/node-crawler

    6,785Vezi pe GitHub↗

    node-crawler is a programmable web crawler for Node.js that manages request queues and automates data extraction. It functions as a rate-limited HTTP client and a headless HTML parser, providing the infrastructure to visit large sets of URLs asynchronously while preventing duplicate processing through task deduplication. The project distinguishes itself through a proxy rotation manager that cycles user agents and proxy servers to bypass access restrictions. It utilizes the HTTP/2 protocol to improve request performance and server compatibility during large-scale scraping operations. The syst

    TypeScriptcheeriocrawlerextract-data
    Vezi pe GitHub↗6,785
    • Vezi toate cele 30 alternative pentru Scrape It→