Awesome GitHub RepositoriesPattern Extraction Utilities

Letta is a framework for building, deploying, and managing autonomous AI agents that maintain persistent state across long-term interactions. It provides a comprehensive suite of primitives for defining agents with configurable personas, modular memory blocks, and tool-use capabilities, enabling them to retain user preferences and conversation history over extended sessions. The platform distinguishes itself through its advanced memory management and orchestration capabilities. It allows agents to autonomously update their own memory, perform retrieval-augmented generation, and coordinate com

Photon is a command-line web crawler designed for security reconnaissance and information gathering. It systematically traverses websites to discover URLs, map domain infrastructure, and identify associated subdomains by retrieving DNS records. The tool distinguishes itself through its ability to perform deep content analysis, including the extraction of sensitive data such as API keys and authentication tokens using user-defined regular expressions. It supports offline inspection by cloning crawled web content to the local filesystem, allowing for structural analysis without additional netwo

SeaTunnel is a distributed data integration engine designed to synchronize structured and unstructured data across diverse sources and sinks. It functions as a multi-engine execution framework that can run data integration tasks across different distributed computing backends to optimize workload performance. The project is distinguished by a visual data pipeline designer for configuring workflows without manual code and a specialized change data capture tool for streaming incremental database updates. It also includes an enrichment pipeline that integrates large language models and embedding

Autoscraper ist eine automatische Web-Scraping-Bibliothek und ein musterbasierter Datenextraktor, der Extraktionsregeln aus Beispieldaten lernt. Er identifiziert und ruft Text, URLs und HTML-Elemente von Webseiten ab, indem er Beispielwerte analysiert, um Datenmuster über verschiedene URLs hinweg zu replizieren. Das System fungiert als Manager für Web-Scraping-Modelle, der es Benutzern ermöglicht, gelernte Regeln zu speichern und neu zu laden, um eine konsistente Datenextraktion aufrechtzuerhalten. Es unterstützt den Export und Import von Scraping-Regeln in ein lokales Dateisystem, um den Trainingsprozess für dieselbe Website nicht wiederholen zu müssen. Die Bibliothek deckt automatisiertes Web-Data-Scraping und Web-Content-Harvesting durch beispielbasiertes Musterlernen und positionsbasierte Elementabfrage ab. Sie kann sowohl spezifische Datenpunkte als auch alle Elemente auf einer Seite abrufen, die den aus den anfänglichen Beispieldaten identifizierten Mustern entsprechen.

Naabu is a port scanner library and tool that probes hosts for open ports using SYN, CONNECT, and UDP methods to identify active services. It functions as a Go library for embedding port scanning into programs, and as a standalone tool that accepts targets as hostnames, IP addresses, CIDR ranges, or ASN numbers. The tool discovers live hosts before scanning, filters ports by range or top lists, and can integrate with Nmap for service version detection. The project distinguishes itself through its SYN-based port probing approach that sends TCP SYN packets and analyzes responses without complet

Ethical-Hacking-Labs is a comprehensive cybersecurity training curriculum and lab suite designed for learning penetration testing, network analysis, and offensive security techniques. It provides a structured environment for practicing the full attack lifecycle, from initial reconnaissance and scanning to exploitation and post-compromise analysis. The project provides instructional materials and guided exercises that cover specific technical domains, including open source intelligence research and network security courseware. It includes a practical workbook for identifying system vulnerabili