Awesome GitHub RepositoriesScripted Crawler Execution

Webmagic is a Java web crawling framework designed for building scalable automated crawlers to download and process large volumes of web pages. It functions as a distributed web crawler and dynamic content crawler, utilizing an XPath HTML parser to locate and extract specific data points from page structures. The framework distinguishes itself through its ability to handle dynamic content by rendering JavaScript and executing asynchronous requests to extract data from non-static pages. It also allows users to define and execute crawler logic via scripting languages, enabling the update of col

SQLiteStudio is an open-source graphical tool for browsing, editing, and managing SQLite database files. It combines a full-featured SQL editor with syntax highlighting, a visual database schema designer for creating entity-relationship diagrams, and a plugin-based extensibility platform that allows adding custom functionality through C/C++, JavaScript, Tcl, or Python. The application distinguishes itself through its multi-language scripting engine, which embeds JavaScript, Tcl, and Python interpreters to enable user-defined functions and scripts within SQL queries. It supports encrypted data

Proxmark3 is an open-source hardware tool for RFID research, analysis, security testing, protocol sniffing, tag cloning, and emulation. It operates across both low-frequency and high-frequency bands, providing a single platform for interacting with a wide range of contactless cards and tags including MIFARE, iCLASS, HID Prox, Hitag, Indala, and T55xx transponders. The tool is built around an FPGA that handles direct RF signal demodulation and modulation, enabling raw waveform capture and decoding. It includes a cryptographic attack framework capable of recovering secret keys through nested, h