Open-source alternatives to Browser

chromedp is a browser automation framework and driver that controls web browsers via the Chrome DevTools Protocol. It functions as a headless browser automation tool and web browser controller, enabling the programmatic management of browser sessions, targets, and network responses through a remote debugging interface. The project provides specialized capabilities for Chrome DevTools Protocol automation, including headless browser testing, web scraping and data extraction, and mobile device emulation. It also supports browser-based visual regression by capturing precise screenshots of web pag

Browserless is a service-oriented platform designed for remote browser automation and headless execution. It provides a distributed infrastructure that manages browser sessions through containerized isolation, allowing users to execute scripts and interact with web content without maintaining local browser state or infrastructure. The platform functions as a remote API and WebSocket-based control layer, enabling stateless HTTP requests for tasks like document generation and real-time browser interaction. It incorporates proxy-based routing to manage traffic signatures and supports the integra

Crawlee-python is a web crawling framework for building scalable scrapers using Python. It serves as a comprehensive tool for web scraping automation, providing a system to extract structured data from websites using both lightweight HTTP requests and headless browser automation. The framework is distinguished by its anti-bot evasion capabilities, which include browser fingerprint impersonation and tiered proxy rotation to bypass detection systems and solve challenges such as Cloudflare. It also incorporates artificial intelligence for autonomous website navigation and schema-based data extra

This project provides a framework for managing multi-agent systems, designed to automate complex software development, infrastructure, and business workflows. It functions as a multi-agent workflow orchestrator that routes tasks to domain-specific workers while maintaining state persistence and infrastructure automation. By leveraging large language models, the system decomposes high-level objectives into actionable plans, ensuring that complex operations are executed with consistency and reliability. The framework distinguishes itself through its hierarchical agent registry and policy-driven

Obscura is a web scraping infrastructure and headless browser server designed for AI agents. It provides a system for AI models to control browser sessions, interact with websites, and extract web data using a WebSocket implementation of the Chrome DevTools Protocol. The project focuses on bot detection evasion by randomizing browser fingerprints, masking native functions, and blocking tracking scripts to mimic human behavior. It further secures identities through a traffic layer that routes network requests via HTTP or SOCKS5 proxies. The system supports large-scale data extraction through

Puppeteer is a JavaScript library for programmatically controlling Chrome and Firefox through the Chrome DevTools Protocol or the WebDriver BiDi protocol. It launches and manages browser instances—typically without a visible user interface—to automate interactions with web pages, enabling navigation, clicking, typing, and data extraction entirely through code. The library distinguishes itself through deep integration with the Chromium embedding layer, allowing fine-grained process configuration with custom flags, permissions, and sandbox policies. It maintains multiple concurrent command stre

This project is a comprehensive resource directory for web data extraction, providing a curated collection of tools and libraries for parsing data, automating browsers, and managing network operations. It serves as a guide for extracting structured information from HTML, XML, JSON, and PDF formats. The toolkit focuses on advanced data collection strategies, including headless browser automation to interact with JavaScript and a suite of network utilities for DNS resolution and WebSocket connections. It specifically covers methods for bypassing bot protections through proxy pool management, us

X-ray is a headless browser web scraper and HTML content crawler designed to extract structured data from websites. It functions as a stream-based data scraper and structured data extractor, using selectors to retrieve text and attributes from HTML as nested objects or arrays. The project includes a request rate controller to manage network traffic through concurrency limits, throttles, and timeouts. It handles dynamic website scraping by rendering JavaScript via a headless browser and performs automated website crawling using breadth-first link following and pagination management. The syste

PhantomJS is a scriptable, headless browser engine based on WebKit that provides a programmatic interface for automating web page interactions. It operates without a graphical user interface, allowing for the execution of JavaScript to navigate pages, manipulate the document object model, and perform functional testing of web applications. The tool distinguishes itself by providing low-level control over the browser rendering lifecycle and network stack. It enables real-time interception and modification of network traffic, alongside the ability to generate visual snapshots and document expor

Maxun is an open-source web scraping and automation platform designed to transform dynamic website content into structured data. By leveraging artificial intelligence to interpret natural language prompts, the system identifies page elements and extracts information without requiring manual selector configuration. It serves as a bridge between raw web content and intelligent workflows, providing structured outputs in formats optimized for large language model ingestion and agent-based applications. The platform distinguishes itself through its ability to handle complex, authenticated, and dyn

Taiko is a browser automation framework and web end-to-end testing library used to perform programmatic user actions and verify application behavior. It functions as a headless browser testing tool capable of simulating real interactions and asserting page states in Chromium and Firefox. The project includes a browser interaction recorder that captures live actions and exports them as executable JavaScript automation scripts. It also serves as a web accessibility auditor, analyzing pages to detect accessibility violations and ensure compliance with inclusive design standards. The framework c

CefSharp is a .NET binding for the Chromium Embedded Framework that allows developers to embed a full web browser into desktop applications. It provides an embedded browser control for rendering HTML, CSS, and JavaScript content within a native host window. The project features a bidirectional JavaScript bridge interface that enables the execution of scripts and the exposure of native host classes and methods to the browser environment. It also includes a headless browser automation tool for executing web tasks and capturing page screenshots without a graphical user interface. The library co

Open Deep Research is an AI-powered web research agent that combines a reasoning model with live web search and data extraction to perform deep, multi-source investigations on any topic. It operates through a dual interface, offering both a command-line tool and a Model Context Protocol server, allowing developers to integrate web capabilities directly into AI agents and coding assistants. The project distinguishes itself by orchestrating an iterative research loop where a reasoning model plans steps, interprets search results, and guides subsequent web interactions. It uses Firecrawl for scr

AnyCrawl is an AI-powered data extractor, automated web crawler, and headless browser orchestrator. It serves as a web content extraction API and a gateway that connects crawling and scraping tools to language models using a standardized API protocol. The project specializes in converting unstructured website content into structured JSON or markdown optimized for AI assistants. It utilizes language models and JSON schemas to pull specific information into validated formats and provides capabilities for AI page summarization and LLM-optimized content extraction. The system manages comprehensi

This project is a comprehensive framework for building and managing autonomous agent systems. It provides a unified architecture for orchestrating multi-agent societies, where specialized agents collaborate through roleplay to decompose and solve complex tasks. The system integrates language models with external environments, enabling agents to perform real-world actions through a standardized tool-calling abstraction layer. The framework distinguishes itself through its focus on iterative reasoning and data reliability. It employs automated feedback loops to refine agent outputs and self-eva

Playwright is a comprehensive browser automation framework designed for end-to-end testing and web workflow automation. It provides a unified API to drive web applications across multiple browser engines, enabling developers to simulate complex user interactions, perform web scraping, and validate application behavior in consistent, isolated environments. The framework distinguishes itself through a web-first testing paradigm that prioritizes stability and resilience. By utilizing an auto-waiting actionability engine and accessibility-tree-based locators, it eliminates common sources of test

Feapder is a Python web crawling framework designed for building scalable data extraction systems. It features a distributed spider engine and a headless browser renderer to execute JavaScript and extract content from dynamic web pages. The system includes a scalable data deduplicator to filter duplicate URLs and records during large-scale operations. A crawler monitoring system tracks the health of active scraping jobs and triggers alerts when system anomalies occur. The framework provides capabilities for task scheduling, web data extraction, and resilient workflows that allow crawling tas

CrawlerTutorial is a comprehensive Python web scraping tutorial and framework designed for extracting data from static and dynamic websites. It functions as a web data extraction pipeline and an HTTP request orchestrator, covering the full lifecycle of scraping applications from initial fetching to final data storage. The project provides specialized guidance on anti-bot bypass techniques and web API reverse engineering. It includes methods for evading browser detection through identity masking and proxy rotation, as well as techniques for identifying hidden API endpoints by analyzing network

This project is a comprehensive educational guide and framework for building web scrapers using Python. It provides a course-based approach to data extraction, combining a Python crawler framework with tutorials on web reverse engineering and network traffic analysis. The project distinguishes itself by covering advanced extraction challenges, including the decryption of obfuscated JavaScript and the bypass of anti-scraping measures. It specifically addresses mobile application scraping through the simulation of user interactions and the interception of network traffic. The capability surfac

This project functions as a Model Context Protocol server and a multi-agent orchestration framework designed to bridge large language models with external data sources and specialized engineering tools. It provides a structured environment for automating software development workflows, enabling models to interact directly with codebases and remote services to perform complex tasks. The system distinguishes itself through a multi-agent orchestration layer that coordinates autonomous assistants to manage shared objectives and multi-step workflows. By utilizing structured task decomposition and

OpenCLI is an AI browser automation framework designed to automate web navigation, data extraction, and repetitive browser tasks. It functions as a browser-based CLI generator that converts website interfaces into command-line interactions by controlling authenticated web browser sessions. The project features a web-to-CLI adapter platform for mapping web elements to programmatic command-line inputs and outputs. It includes a browser profile manager to organize and switch between isolated session profiles to maintain different user identities. The toolkit provides capabilities for web conten

This project is a Python web scraping library and automated data collection suite. It provides tools for extracting structured data from websites, implementing web crawlers to navigate site links, and parsing HTML DOM structures to isolate specific elements and attributes. The toolkit includes a pipeline for processing unstructured text and cleaning raw web content to extract meaningful information. It also features capabilities for image data extraction and the integration of external APIs to retrieve structured data from remote endpoints. The system covers broad capability areas including

PySpider is a Python web crawling framework designed for automated data extraction. It provides a pipeline for periodically fetching web content, processing HTML, and persisting scraped information into database backends. The system features a web-based management interface for editing scraping scripts, monitoring task progress, and reviewing collected data. It includes a headless browser JavaScript renderer to capture rendered HTML from dynamic web pages and a distributed architecture that uses message queues to scale crawling workloads across multiple nodes. The framework also covers task

Portia is a containerized scraping platform and visual web scraper that enables no-code data extraction. It serves as a Scrapy visual scraping tool and spider generator, allowing users to design and deploy web scrapers through a graphical interface instead of writing manual selector code. The system distinguishes itself by converting visual web page annotations into executable Scrapy spider code and structured JSON specifications. This visual-to-code mapping allows users to define scraping logic and extraction rules through a point-and-click interface, which can then be exported for use in ex

OpenBrowser is an AI web agent toolkit and automation framework designed to translate natural language instructions into executable browser workflows. It functions as a headless browser controller and orchestrator, enabling the creation of autonomous agents that navigate websites, interact with elements, and extract data using plain English commands. The system features a sandboxed execution environment that utilizes domain whitelists and memory limits to ensure secure web interaction. It distinguishes itself through a command-line interface for triggering autonomous tasks with configurable m

Eko is a framework for designing and deploying agentic workflows, featuring an LLM agent workflow orchestrator and a browser automation engine. It provides a server-side process manager for executing system-level operations and managing local files, alongside a human-in-the-loop agent controller for manual oversight and direction during automated decision processes. The system coordinates multi-agent collaboration through role-based partitioning and workflow orchestration, dividing complex tasks into distinct roles and managing execution handoffs. It integrates the Model Context Protocol to s

This project is a Python web scraping tutorial and framework designed for building automated data extraction tools and web crawlers. It provides a structured approach to navigating websites and persisting scraped data to databases. The project includes a toolset for web API analysis, focusing on reverse engineering obfuscated API requests and inspecting network traffic to extract structured data. It also covers optical character recognition workflows to convert visual text within images into machine-readable strings. The framework covers capabilities for headless browser automation to handle