Open-source alternatives to Google Maps Scraper

Ripme is a batch media downloader and web media scraper designed for extracting images and videos from image-hosting platforms and social media sites. It functions as an image gallery downloader and a network client capable of retrieving full albums and paginated content. The project includes a custom media ripper framework that allows for the definition of new extraction rules to support websites lacking native support. It features a proxy-enabled network layer for routing requests through HTTP or SOCKS servers and supports session-based content retrieval using authentication cookies and cus

urllib3 is a Python HTTP client library used to send network requests and receive responses. It provides core components for managing HTTP connection pools, routing traffic through proxies, validating TLS certificates, and executing automatic request retries. The library focuses on network reliability and efficiency by maintaining a system that reuses established connections to multiple hosts to reduce latency. It ensures secure communication through client-side certificate verification and handles transient network errors using policy-based retry logic. The project covers broad networking c

Stealth is a distributed automation engine designed for headless browser orchestration, web scraping, and decentralized network coordination. It functions as a peer-to-peer framework that enables multiple nodes to align automated tasks, share cached web resources, and route traffic through a decentralized proxy architecture. By abstracting the underlying browser implementation, the project provides a unified interface for executing navigation and data extraction workflows across diverse environments. The platform distinguishes itself through its ability to synchronize browser sessions and aut

Goutte is a PHP web scraper and DOM crawler designed for extracting data from websites. It functions as an HTTP client wrapper that enables the retrieval of web pages and the parsing of HTML content. The project provides a web form automator to programmatically fill and submit HTML forms to remote servers. It also includes a mechanism for automated website crawling by following links to discover and archive web content. The system supports stateful session management to maintain cookies and headers across requests. It further covers HTML data extraction through DOM-based element selection an

pipet is a command-line tool that turns web scraping into a piped data flow through Unix filters. It provides a set of specialized scrapers — for CSS selector extraction, headless browser JavaScript rendering, JSON API querying, and change monitoring — each outputting structured data that can be transformed by chaining additional commands. The tool uses declarative selectors (CSS and JSON path expressions) to define what to extract, automatically follows pagination links to collect data across multiple pages, and serializes results into JSON, custom-delimited text, or rendered templates. It c

FlareSolverr is a proxy server designed to provide programmatic access to websites protected by automated security challenges and firewall restrictions. It functions by orchestrating headless browser instances to render web pages, execute JavaScript, and retrieve the necessary cookies and content required to bypass common security hurdles. The service distinguishes itself by maintaining persistent browser sessions in memory, which allows for the reuse of authenticated states across multiple requests. It integrates with external captcha resolution services to handle interactive security challe

Ghost-Downloader-3 is a multithreaded download manager and HTTP file downloader designed to accelerate file transfers. It functions as a browser-integrated download client, bridging web extensions to a local application to automate task capture and transfer. The system utilizes a multithreaded transfer engine that employs parallel chunking to maximize bandwidth. It distinguishes itself through a concurrent transfer orchestrator that manages simultaneous download queues, provides dynamic bandwidth throttling, and supports proxy server routing to manage network paths. The application includes

PCAPdroid is an Android network traffic analyzer and packet capture tool that operates without requiring root access. It functions as a VPN-based firewall and network controller, capable of recording traffic in PCAPng format and blocking connections to specific domains or malicious hosts. The project distinguishes itself through a proxy-based system for decrypting TLS traffic and routing device network traffic through SOCKS5 proxies or the Tor network. It further allows for the modification of live HTTP requests and responses via custom scripts. Its capabilities cover application connection

This project is an MCP browser automation server that connects large language models to headless cloud browsers. It functions as an autonomous web workflow engine and an LLM web agent interface, enabling the translation of natural language instructions into browser actions and structured data retrieval. The system distinguishes itself through a managed headless browser cloud API that supports concurrent Chromium sessions with integrated stealth modes, CAPTCHA solving, and proxy traffic routing. It utilizes self-healing element selection to maintain automation resilience when page structures c

Colly is a web scraping framework and concurrent crawler written in Go. It provides a system for traversing web pages, following links, and extracting structured data from HTML and XML documents. The framework includes a distributed scraping engine designed to spread data collection tasks across multiple instances to increase throughput. It ensures compliance with website owner policies by automatically reading and respecting robots.txt files. The system manages request lifecycles through domain-based rate limiting, concurrency controls, and session management via a stateful cookie jar. It s

Livestreamer is a command-line stream extractor and modular streaming client designed to resolve hidden media source URLs from online services. It functions as a media pipeline utility that extracts direct media URLs and binary data for local saving, specialized processing, or routing into external media players. The project uses a plugin-based system with custom heuristic logic to extend service support and override default capabilities. It includes a network proxy router to bypass regional blocks and network restrictions through configurable HTTP and HTTPS proxies, and manages authenticatio

This project is a community-curated directory of open-source software designed for deployment in private server environments and home labs. It serves as a comprehensive resource for discovering independent, self-hosted alternatives to mainstream cloud services, enabling users to maintain full data ownership and control over their digital infrastructure. The directory is structured through a hierarchical taxonomy that organizes a vast collection of applications into logical categories, ranging from media management and data analytics to private communication and team productivity tools. It dis

This project is a comprehensive educational guide and framework for building web scrapers using Python. It provides a course-based approach to data extraction, combining a Python crawler framework with tutorials on web reverse engineering and network traffic analysis. The project distinguishes itself by covering advanced extraction challenges, including the decryption of obfuscated JavaScript and the bypass of anti-scraping measures. It specifically addresses mobile application scraping through the simulation of user interactions and the interception of network traffic. The capability surfac

This project is an Android RPA framework designed for automating user interfaces and system tasks on rooted Android devices using Python and ADB. It provides a suite of tools for rooted device management, allowing for programmatic control of system settings, application lifecycles, and shell command execution via a remote API. The framework distinguishes itself through a combination of dynamic instrumentation and AI integration. It can inject scripts into running processes to hook Java interfaces and modifies application behavior in real time. Additionally, it supports large language model in

h8mail is an open-source intelligence tool for searching leaked credentials and compromised accounts across remote APIs and local data dumps. It functions as a credential leak hunter and email reconnaissance framework designed to identify exposed passwords and sensitive information using usernames, domains, IP addresses, and email hashes. The tool distinguishes itself through a recursive target expansion system that feeds newly discovered email addresses back into the search queue to broaden the scope of investigations. It also includes a local breach data parser that employs multiprocessing

Nettacker is an automated penetration testing framework designed to orchestrate reconnaissance, port scanning, and vulnerability detection. It functions as a network reconnaissance tool and vulnerability scanner that identifies open ports, fingerprints services, and checks systems against databases of known security flaws. The framework distinguishes itself by combining a web application crawler for discovering hidden paths via fuzzing with a vulnerability management system that persists scan results in a database to track historical assessments. It also includes specialized capabilities for

Syncthing macOS is a native application bundle that wraps the Syncthing binary to provide decentralized, peer-to-peer file synchronization on macOS. It enables bidirectional data replication across multiple devices using a block exchange protocol, allowing files and folders to stay consistent without relying on a central cloud server. The project distinguishes itself by integrating the cross-platform synchronization engine into the macOS environment as a background service. It provides a system tray interface for quick access to the management GUI and supports automatic application startup th

requests-html is a Python HTML parsing library and web scraping framework. It functions as an asynchronous HTTP client and a JavaScript rendering engine designed to fetch and parse web pages for structured data extraction. The project integrates a headless browser to execute JavaScript, allowing it to retrieve dynamically generated content that standard HTML parsers cannot see. It provides tools for automated data extraction using CSS selectors and XPath expressions to isolate specific text or attributes from HTML structures. The framework covers network operations including asynchronous pag

This project is an OpenWrt proxy client and network traffic orchestrator designed to tunnel internet traffic through remote proxy nodes. It functions as a system for managing proxy subscriptions, preventing DNS leaks, and directing traffic flow via an automatic failover gateway. The software distinguishes itself through a subscription manager that imports and synchronizes remote server lists via URLs and QR codes. It features device-specific access controls that allow the application of unique proxy rules, domain whitelists, and time-based schedules to individual network devices. The capabil

Pholcus is a distributed web crawler framework written in Go designed for high-concurrency data extraction. It functions as a distributed crawling orchestrator and dynamic data extraction engine, utilizing a server-client architecture to coordinate tasks across multiple nodes. The system integrates a headless browser engine to render dynamic content and execute JavaScript, allowing it to extract data from single-page applications. It features a web-based management interface for configuring spider parameters and monitoring execution progress, alongside the ability to update extraction rules v

wool_scripts is a collection of HTTP proxy rule sets, network traffic filters, and automation scripts designed for mobile applications. It functions as an application proxy router to reroute specific traffic and AI model requests through proxy servers to change IP addresses. The project provides a system for intercepting network requests to remove embedded advertisements from mobile applications and mini-programs. It also includes filters to suppress operating system update notifications and automatic software downloads by blocking requests to update servers. Beyond traffic filtering, the re

Spider is a web-based platform designed for automated data extraction, providing a centralized framework to collect, process, and route structured information from websites. It functions as a comprehensive pipeline that manages the entire lifecycle of data gathering, from initial configuration to final storage in external databases or message queues. The platform distinguishes itself through a visual configuration interface that allows users to define extraction rules and manage scraping templates without writing custom code. It supports both static and dynamic content retrieval by integratin

This project is a Go-based HTTP proxy server designed as a censorship circumvention tool. It functions as an upstream proxy manager and SOCKS5 tunneling gateway that routes network traffic between clients and destination servers to bypass network restrictions. The system differentiates itself through automated proxy routing, which detects unreachable websites and automatically switches traffic between direct access and a pool of parent proxies. It includes a PAC file generator to produce proxy auto-config files for browsers and integrates SSH tunneling to establish secure remote sockets. Bro

Scrapy is a comprehensive framework designed for automated web data extraction and large-scale crawling. It operates on an asynchronous, event-driven engine that manages non-blocking network requests and data processing tasks, allowing for the efficient retrieval of structured information from web documents using path-based selectors. The system distinguishes itself through a highly modular architecture that supports complex data collection workflows. Users can implement custom middleware and signal handlers to intercept and modify request flows, while a priority-based scheduler manages concu

TelegramGroup is a comprehensive automation framework designed for managing multiple messaging accounts and orchestrating complex administrative workflows. It functions by emulating client-side sessions to interact with platform APIs, enabling centralized control over user profiles, persistent session data, and distributed network routing through proxy infrastructure. The platform distinguishes itself through its modular architecture, which supports independent plugins for tasks such as artificial intelligence integration, content mirroring, and automated community moderation. It provides a s

Clashfree is a network traffic routing platform designed to facilitate access to restricted online resources and digital services. It functions as a proxy configuration management tool that enables users to route internet traffic through encrypted tunnels, effectively bypassing regional access restrictions. The system provides a centralized way to manage network proxy connections and organize multiple routing profiles across various environments. The project distinguishes itself by providing automated subscription services that distribute daily updated proxy node lists and configuration files

Crawlab is a distributed web scraping platform designed to centralize the management, deployment, and execution of large-scale data extraction tasks. It functions as a control plane that orchestrates scraping scripts and automated workflows across multiple nodes, providing a unified environment for managing complex data collection operations. The platform distinguishes itself through a distributed architecture that coordinates worker nodes via a central master, utilizing real-time communication to maintain oversight of all active processes. It ensures operational consistency by isolating task