scrapinghub/splash
Splash
Splash est une API HTTP de navigateur headless et un moteur de rendu JavaScript conçu pour convertir le contenu web dynamique en HTML statique ou en images. Il fonctionne comme un service de navigateur scriptable en Lua qui expose des capacités d'automatisation et de rendu via une interface RESTful pour l'extraction de données programmatique.
Le service se distingue en permettant l'exécution de scripts Lua personnalisés pour automatiser des séquences d'interaction utilisateur complexes et la navigation sur les pages. Il offre la possibilité de changer de moteur de rendu par requête pour vérifier la compatibilité entre navigateurs et la cohérence visuelle.
La plateforme couvre un large éventail de capacités, incluant l'interaction web automatisée via des événements clavier et souris simulés, ainsi que l'interception du trafic réseau pour surveiller ou modifier les requêtes et réponses. Elle prend également en charge l'extraction DOM via des sélecteurs CSS et l'exportation de journaux réseau pour l'analyse du trafic.
Features
- Headless Browser Automation - Implements a headless browser automation system for programmatically simulating user interactions and extracting data.
- Browser Automation APIs - Exposes browser automation and rendering capabilities via a stateless REST API for remote execution.
- JavaScript Rendering - Converts JavaScript-heavy websites into static HTML for programmatic data extraction.
- Web Page Rendering - Transforms dynamic web pages into static HTML, PNG, or JPEG files via an HTTP API.
- Embedded Lua Scripting - Integrates an embedded Lua engine to define custom interaction sequences and page logic.
- Lua Script Execution Environments - Executes custom Lua scripts to control browser behavior and automate complex interaction sequences.
- Browser Automation Scripts - Provides a Lua-scriptable environment to automate complex user interaction sequences and page navigation.
- Element Interaction - Simulates user input by sending keyboard events, performing mouse clicks, and filling out web forms.
- Headless Rendering Engines - Uses a headless rendering engine to execute JavaScript and generate final HTML or image snapshots.
- Web Automation Frameworks - Simulates user actions like clicking buttons and filling forms through a programmatic HTTP API.
- REST APIs - Exposes headless browser functions through a RESTful HTTP API for programmatic control and data extraction.
- Web Scraping - Extracts structured data from modern web applications by controlling a headless browser.
- CSS Selector - Provides CSS selectors to extract specific attributes and node content from the rendered DOM.
- Browser Screenshot Capture - Takes snapshots of full webpages or specific regions in common image formats for visual verification.
- Network Request Interceptors - Includes tools to monitor, modify, or block outgoing network requests and incoming responses.
- Traffic Interception Tools - Monitors or modifies outgoing requests and incoming responses to block resources based on headers.
- Network Traffic Logging - Provides the ability to capture and format network requests and responses for debugging page load sequences.
- Visual Consistency Testing - Enables visual verification of layout and consistency by capturing snapshots across different rendering engines.
- Data Extractions - Retrieves specific content from the rendered page using CSS selectors to access DOM node attributes.
- Network Request Interception - Monitors and modifies network traffic to block specific resources or capture detailed logs during page loads.
- Request-Browser Toggles - Allows switching between different rendering backends on a per-request basis to verify cross-browser compatibility.
- Browser Engines - Lightweight JavaScript rendering service with an HTTP API.
