Open-source alternatives to Physical Web

Trafilatura is a Python library and command-line tool for extracting clean, structured text and metadata from web pages. It downloads HTML content, identifies the main body of text, and strips away navigation, ads, and other boilerplate, returning the core article content along with fields like title, author, date, and URL. The tool can also extract user comments and test whether a page contains extractable text, making it a general-purpose web text extraction library. What distinguishes Trafilatura from simpler extractors is its configurable extraction pipeline, which offers high-speed, high

This project is a comprehensive resource directory for web data extraction, providing a curated collection of tools and libraries for parsing data, automating browsers, and managing network operations. It serves as a guide for extracting structured information from HTML, XML, JSON, and PDF formats. The toolkit focuses on advanced data collection strategies, including headless browser automation to interact with JavaScript and a suite of network utilities for DNS resolution and WebSocket connections. It specifically covers methods for bypassing bot protections through proxy pool management, us

ESP32-DIV is a handheld wireless pentesting platform designed for analyzing and disrupting a wide range of wireless protocols. It functions as a multi-band radio analyzer, RFID and NFC tag manipulator, and GPS wardriving logger, providing a unified interface for security auditing and signal research. The project distinguishes itself through a modular radio abstraction that allows switching between Wi-Fi, BLE, Sub-GHz, RFID/NFC, and infrared hardware modules. It features a touch-driven TFT interface for navigating toolsets and managing signal profiles, as well as the ability to emulate Bluetoo

Reth is a modular, high-performance Ethereum execution layer client written in Rust. It serves as a full Ethereum node that syncs, validates, and serves blockchain data, functioning as an archive node implementation, a high-throughput RPC node server, and a snapshot sync tool. The project is built around a modular component architecture that allows assembling custom node behavior by swapping independent Rust crates for consensus, execution, mempool, and networking. The client distinguishes itself through a staged sync pipeline that downloads headers and bodies online before processing the res

This repository is a comprehensive collection of reference implementations and sample libraries for the Universal Windows Platform. It provides practical examples of how to use Windows Runtime APIs to build cross-device applications, including detailed guidance on XAML-based declarative user interfaces and DirectX-integrated rendering. The project distinguishes itself by providing a wide array of hardware integration suites, covering low-level communication with USB, Serial, I2C, SPI, and GPIO peripherals. It includes specialized implementations for mixed reality holographic rendering, advanc

This library is a framework for managing Bluetooth Low Energy connections, device scanning, and data exchange between Android applications and peripheral hardware. It provides a development toolkit for implementing service operations, notification handling, and broadcast packet parsing to facilitate communication with external sensors and equipment. The library distinguishes itself by maintaining multiple concurrent peripheral connections with automatic retry logic and state management to ensure stable communication sessions. It handles the complexities of hardware interaction by serializing

ipscan is a cross-platform network auditor and Java-based network scanner used for discovering networked devices and exporting scan results. It functions as an IP address range scanner and a TCP/UDP port scanner to identify active hosts and open services on a network. The application features a plugin-based fetcher architecture that allows the integration of custom data collection fetchers to retrieve specific hardware or software information from discovered hosts. Its broader capabilities include local network discovery, network port auditing, and the creation of network asset inventories.

This project is a multi-format barcode library designed to encode and decode one-dimensional and two-dimensional barcodes across multiple programming languages. It functions as a cross-platform image processor that analyzes visual data to detect, locate, and extract information from patterns in diverse environments, while also providing a standard for mapping structured data into machine-readable formats. The library distinguishes itself through advanced image processing techniques that ensure reliability in real-world conditions. It employs pattern-matching detectors to identify geometric fi

Snapdrop is a web-based local file sharing tool and progressive web app designed for transferring files between devices on the same local network. It functions as an end-to-end encrypted transfer tool that allows users to move data across different devices and operating systems without manual configuration. The service supports self-hosting through a containerized deployment model, allowing users to run private instances of the file sharing service on their own infrastructure. This ensures that data transfers remain within a private local network. The system uses a signaling server for local

Valetudo is a custom firmware project for robot vacuums that enables local-only control, removing dependencies on cloud servers and protecting user privacy. It replaces proprietary vendor binaries with open source software to ensure that data, including floor plans and images, is not uploaded to external clouds. The project distinguishes itself by providing a rooted firmware installation process that prevents forced vendor updates. It implements a standardized control interface across different hardware brands and utilizes an MQTT-based message bus to facilitate integration with open source a

Snapdrop is a browser-based peer-to-peer file sharing tool that transfers files directly between devices on the same local network without uploading to any intermediate server. It operates as a progressive web app, meaning it works across devices without installation, and uses end-to-end encryption to ensure only the sender and receiver can read the content being transferred. The application establishes direct browser-to-browser connections using WebRTC, splitting large files into chunks for real-time streaming with progress tracking. It discovers nearby devices on the same subnet through loc

Opendrop is a Python implementation of the AirDrop protocol used for transferring files and links between devices over wireless connections. It functions as a peer-to-peer wireless file transfer system and a tool for automating the exchange of network passwords using specialized handshake mechanisms. The project enables the discovery of nearby hardware by scanning for Bluetooth Low Energy advertisement packets and searching local subnets for specific service signatures. It establishes direct communication links through monitor mode and raw frame injection to move data without requiring a stan

Reticulum is a decentralized networking stack that enables encrypted, peer-to-peer communication over diverse physical mediums without relying on central infrastructure or IP protocols. It uses self-sovereign cryptographic identities for routing and authentication, replacing traditional IP addresses with collision-free globally unique addresses that require no central coordination. Every packet is encrypted by default using ephemeral key exchanges with forward secrecy, and unencrypted traffic is dropped as invalid. The stack unifies heterogeneous transport mediums—including LoRa radio, packet

NearDrop is a cross-platform file transfer tool and a Google Quick Share client for macOS. It enables the movement of files and documents between Android devices and macOS computers over a local network. The application implements the Google Quick Share protocol to receive files from Android phones on a Mac without using cloud storage. It includes a utility for establishing secure connections between devices via scannable QR codes when automatic discovery fails. The software manages local network data sharing through device discovery and secure pairing mechanisms. It facilitates the transfer

Zotero is reference management software designed for collecting, organizing, and citing bibliographic research sources and digital documents for academic work. It functions as a web bibliographic collector, a citation generator, and a collaborative research platform. The system integrates tools for capturing metadata and archiving web pages into a centralized research library. It provides a specialized environment for reading and marking up PDF and EPUB files with highlights and notes linked directly to research sources. The software covers a broad range of capabilities including bibliograph

ShareDrop is a browser-based platform designed for direct, peer-to-peer file exchange between devices. It facilitates data transfer by establishing encrypted connections directly between browsers, ensuring that files are transmitted without being stored on central servers. The system identifies nearby devices on the same local network to enable immediate, configuration-free file sharing. For remote participants, it provides the ability to create private, temporary web rooms that use unique, ephemeral addresses to establish secure connections across different networks. The project coordinates

Ethical-Hacking-Labs is a comprehensive cybersecurity training curriculum and lab suite designed for learning penetration testing, network analysis, and offensive security techniques. It provides a structured environment for practicing the full attack lifecycle, from initial reconnaissance and scanning to exploitation and post-compromise analysis. The project provides instructional materials and guided exercises that cover specific technical domains, including open source intelligence research and network security courseware. It includes a practical workbook for identifying system vulnerabili

Crawlee-python is a web crawling framework for building scalable scrapers using Python. It serves as a comprehensive tool for web scraping automation, providing a system to extract structured data from websites using both lightweight HTTP requests and headless browser automation. The framework is distinguished by its anti-bot evasion capabilities, which include browser fingerprint impersonation and tiered proxy rotation to bypass detection systems and solve challenges such as Cloudflare. It also incorporates artificial intelligence for autonomous website navigation and schema-based data extra

PhoneSploit-Pro is an Android exploitation and remote administration toolkit. It provides a system for remotely controlling Android devices and extracting data using the Android Debug Bridge protocol. The project integrates with Metasploit to automate the creation and installation of payloads, establishing persistent command-and-control sessions. It also includes a network device scanner to identify active Android hosts on local networks by probing TCP ports. The toolkit covers a broad range of administrative and forensic capabilities, including remote shell execution, application management

wlan-sec-test-tool is a collection of specialized wireless security tools designed for scanning access points, auditing WPA, WPA2, and WPA3 security, and performing automated password cracking and connection testing. It functions as a wireless network scanner, a password cracker, and a security auditor to evaluate wireless protocol vulnerabilities. The tool differentiates itself through the use of a concurrent connection tester that executes multiple simultaneous connection attempts. It utilizes custom password lists and dictionary-based iteration to verify wireless network security and deter

blinker-js is an IoT device communication framework and Node.js library designed to manage bidirectional data exchange between embedded hardware and remote servers or local networks. It functions as an MQTT hardware integration tool that links single-board computers to IoT clouds for remote management and control. The project provides a virtual device widget manager to synchronize hardware state with cloud-based user interface components. This system allows for the mapping of remote UI identifiers to internal software logic, enabling remote IoT control through virtual buttons and input fields

chromedp is a browser automation framework and driver that controls web browsers via the Chrome DevTools Protocol. It functions as a headless browser automation tool and web browser controller, enabling the programmatic management of browser sessions, targets, and network responses through a remote debugging interface. The project provides specialized capabilities for Chrome DevTools Protocol automation, including headless browser testing, web scraping and data extraction, and mobile device emulation. It also supports browser-based visual regression by capturing precise screenshots of web pag

Sharedrop is a browser-based file drop application for transferring files directly between devices without requiring software installation or account registration. It uses WebRTC to establish peer-to-peer data channels, allowing files to be exchanged between different operating systems without routing traffic through a central server. The project provides mechanisms for both local and remote connectivity. It identifies active users sharing the same public IP address for immediate local device discovery and allows the creation of private sharing rooms via unique session links for users on diff

wifijammer is a wireless network utility designed to identify wireless targets and disrupt active network connections through management frame injection. It functions as a Wi-Fi deauthentication tool and wireless scanning utility that injects frames into the air to force devices to drop their network links. The software enables targeted disruption by focusing on specific hardware addresses across multiple radio frequencies. It employs a scanning capability that hops through Wi-Fi channels to identify active access points and connected clients within range. The project covers capabilities for

jellyfin-android is an Android media server client used to stream movies, television shows, and music from a remote server to mobile devices. It functions as a cross-platform media player that supports direct streaming, remote transcoding for incompatible formats, and the integration of external subtitles. The application serves as a Chromecast media controller, allowing users to cast server-hosted content to compatible televisions and speakers. It also acts as a digital library viewer for browsing photo collections and reading ebooks, comics, and magazines stored on a remote server. Beyond

Browser Control is a command-line browser control tool written in Rust that communicates directly with browser instances using low-level remote debugging protocol commands over web sockets. It provides a stateless command-line interface that processes single terminal commands and outputs compact text or JSON for automated scripts and coding agents. The project manages real browser process lifecycles, connects to cloud-hosted sessions or local instances, and runs multi-step automation scripts from files or piped input streams. The utility features stable reference element mapping that generate

Defuddle is a command line web parser and content extractor designed to isolate the primary article body from web pages and convert the result into standardized markdown. It functions as a content cleaner that removes layout clutter, such as sidebars and headers, to retrieve the main text and associated metadata. The tool provides a terminal interface that processes content from remote URLs, local files, or piped HTML streams. It supports custom content targeting, allowing users to specify CSS selectors to manually define the main content area when automatic detection is insufficient. The sy