Awesome GitHub RepositoriesHTTP Metadata Accessors

Nushell is a cross-platform shell and programming language designed to treat all input and output as structured data rather than raw text streams. By enforcing data types and command signatures, it provides a consistent environment for building robust, pipeline-oriented workflows. The shell allows users to chain commands that pass structured objects between stages, enabling complex data processing and automation tasks that remain predictable across different operating systems. What distinguishes the project is its focus on interactive data exploration and modular extensibility. Users can quer

Crawlee is a web scraping framework designed for building scalable, reliable, and distributed data extraction pipelines. It provides a unified interface for managing headless browser automation and lightweight HTTP requests, allowing developers to handle complex web navigation, dynamic content rendering, and large-scale data collection within a single, modular architecture. The project distinguishes itself through its resource-aware concurrency controller, which dynamically scales task execution based on real-time CPU and memory usage to prevent host machine exhaustion. It also features a rob

Gowitness is a system for rendering web interfaces at scale to capture visual snapshots, HTTP metadata, and network scan results. It functions as a headless browser screenshot tool and a web surface mapper used to identify and visually document the attack surface of network ranges and URL lists. The tool includes a screenshot gallery server that provides a web-based interface for browsing, filtering, and managing a database of captures. It specifically serves as an Nmap target visualizer, parsing network scan results to automatically capture screenshots of discovered web services. Capabiliti