Projects sharing features with TopList

LearnPython is a programming tutorial consisting of a collection of practical code examples used to demonstrate Python language features and programming patterns. It serves as a comprehensive learning resource that implements core language concepts through functional code. The project provides specialized guides and samples covering several key domains. These include asynchronous network programming with event loops and coroutines, data visualization using numerical datasets for 2D and 3D plots, and web scraping for fetching content and automating login flows. It also features instructions on

This project is a Hacker News clone built with Vue.js. It serves as a demonstration of a server-side rendered application and a progressive web app that supports offline access and mobile installation. The application showcases the implementation of centralized state management for coordinating global data flow and a routing system that features route-level code splitting and animated view transitions. It covers broader capabilities in frontend route optimization, server-side rendering for improved initial load speeds, and the use of service workers and app manifests to enable offline functi

Horizon is an AI-powered news aggregation system designed to build custom pipelines that fetch, filter, and enrich information from diverse web sources. It utilizes large language models to automate information filtering, scoring content to remove noise and highlight high-value stories. The system integrates the Model Context Protocol to expose pipeline stages as tools for external AI assistants. It employs a unified adapter to standardize diverse AI model providers for consistent content scoring and summarization tasks. The pipeline aggregates data from RSS feeds, social platforms, financia

This project is a Node.js web scraping framework designed to automate data extraction through a programmatic workflow of requests, parsing, and document interaction. It functions as a headless web crawler, an HTTP request manager, and a DOM parser and extractor. The framework distinguishes itself by combining a JavaScript execution engine to interact with dynamic content and a hybrid selection system that utilizes both CSS and XPath selectors. It includes specialized middleware for proxy rotation and cookie-jar session management to maintain authenticated states and manage automated traffic.

HackerNews-React-Native is a cross-platform mobile client and news aggregator designed for iOS and Android. It provides a mobile interface for browsing paginated news stories and reading nested community discussions by fetching data from the Hacker News API. The application includes an embedded web viewer to render external articles and source pages directly within the interface. It implements a hierarchical discussion system to display nested comment threads. The project is built using a React Native codebase and utilizes a modular UI composition. It incorporates list virtualization for sto

node-crawler is a programmable web crawler for Node.js that manages request queues and automates data extraction. It functions as a rate-limited HTTP client and a headless HTML parser, providing the infrastructure to visit large sets of URLs asynchronously while preventing duplicate processing through task deduplication. The project distinguishes itself through a proxy rotation manager that cycles user agents and proxy servers to bypass access restrictions. It utilizes the HTTP/2 protocol to improve request performance and server compatibility during large-scale scraping operations. The syst

DotnetSpider is a .NET web crawling framework and C# data extraction tool designed for automated web page discovery and the retrieval of structured data from the internet at scale. It functions as a high-level web scraping library for collecting information from various websites. The framework provides capabilities for automated web crawling and large-scale data scraping. It enables web content extraction to facilitate the creation of local databases or the analysis of online information through programmatic web automation within the .NET ecosystem. The system utilizes a pipeline-based data

X-Ray is a web scraping framework and asynchronous web crawler designed to extract structured data from websites. It functions as an HTML data extractor that transforms raw page content into a defined schema using CSS-style selectors. The project implements a headless browser crawler capable of executing JavaScript to render dynamic content. It handles website content discovery through a breadth-first crawling strategy and automatic pagination discovery to traverse multi-page result sets. The framework manages web data pipelines using a concurrency-limited request queue and request rate cont

This project is a manga source extension repository and content aggregator. It functions as an HTTP content scraping engine that retrieves images and metadata from external provider websites by parsing HTML and making network requests to display digital manga within a unified reader. The system utilizes a JSON extension repository to allow reader applications to discover and install third-party content providers. It employs an interface-based plugin framework that defines a common set of methods to ensure external sources remain compatible with a standardized internal format. The project cov

Crawler4j is a multi-threaded Java web crawler and spider designed for high-volume web traversal and content extraction. It functions as a polite crawling framework that enables the discovery and indexing of HTML and binary content across multiple websites. The project distinguishes itself through a persistent crawling model that serializes session state to local storage, allowing the engine to resume indexing after a crash or interruption. It includes a politeness controller to regulate request frequency and delays, preventing server overloading and IP blocking. The system covers a broad ra

This repository is a collection of Java backend projects, encompassing a web application suite, a real-time chat server, a web crawler, and an authentication system. It provides implementations for building server-side logic and business rules to process data and handle web requests. The project includes a networked messaging system for instant bidirectional communication and an automated web scraping engine that parses HTML to aggregate data into local storage. It also features a security framework for managing user permissions and generating visual captchas to prevent automated bot access.

AnyCrawl is an AI-powered data extractor, automated web crawler, and headless browser orchestrator. It serves as a web content extraction API and a gateway that connects crawling and scraping tools to language models using a standardized API protocol. The project specializes in converting unstructured website content into structured JSON or markdown optimized for AI assistants. It utilizes language models and JSON schemas to pull specific information into validated formats and provides capabilities for AI page summarization and LLM-optimized content extraction. The system manages comprehensi

PythonSpiderNotes is a comprehensive instructional resource and framework for building web crawlers and extracting data using the Python programming language. It provides a set of methods for parsing unstructured HTML and JSON data into structured formats for persistent storage. The project includes detailed guides and tutorials on browser automation for retrieving dynamic content, as well as a framework for data extraction. It specifically covers anti-bot bypass techniques, such as rotating proxies and spoofing headers, to avoid IP blocks and detection systems. The capability surface extend

Obscura is a web scraping infrastructure and headless browser server designed for AI agents. It provides a system for AI models to control browser sessions, interact with websites, and extract web data using a WebSocket implementation of the Chrome DevTools Protocol. The project focuses on bot detection evasion by randomizing browser fingerprints, masking native functions, and blocking tracking scripts to mimic human behavior. It further secures identities through a traffic layer that routes network requests via HTTP or SOCKS5 proxies. The system supports large-scale data extraction through

myGPTReader is a suite of large language model applications including a chat interface, a document analysis tool, and a news aggregator. The system focuses on extracting information from digital files and web content to enable conversational analysis and automated content condensation. The project features a prompt template manager to structure conversation flows and increase response accuracy. It also includes a multilingual voice chat client that integrates speech-to-text and text-to-speech for real-time interactive tutoring and language practice. The platform covers broader capabilities i

leek-fund is a financial market extension for VSCode that enables the monitoring of real-time pricing data for stocks, funds, and futures directly within the editor status bar. It functions as a market data dashboard and portfolio tracker, allowing users to manage investment portfolios and track global exchange price fluctuations without leaving their workspace. The project distinguishes itself by integrating an AI market analyst to process investment information and interpret market trends. It further incorporates a financial news aggregator that streams real-time market alerts and social up

This project is a web-based manga and novel downloader and multi-site web scraper designed to extract images and text from diverse media platforms. It functions as a digital media archiver and EPUB e-book generator, using a plugin-based crawler architecture with site-specific scripts to define how content is extracted from various international websites. The system distinguishes itself through authenticated web crawling, using browser cookie simulation to access restricted or member-only content. It includes specialized capabilities for digital comic archiving, which organizes image sequences

RED_HAWK is a penetration testing framework and reconnaissance suite designed for information gathering and vulnerability assessment. It provides a toolkit for infrastructure reconnaissance, technology stack detection, automated web spidering, and security scanning. The project distinguishes itself through a multi-stage reconnaissance pipeline that maps attack surfaces. This includes DNS-based infrastructure mapping to resolve network layouts and pattern-based detection to identify specific content management systems and server stacks. The system covers a broad range of capabilities includin

WeChatRobot is a framework providing programmatic interfaces for automating messages and interactions within the WeChat ecosystem. It functions as a bot framework that enables the automation of sending and receiving messages by intercepting system processes. The project includes specialized tools for intercepting system calls and utilizing memory-based application hooking to automate workflows, including support for Enterprise WeChat. It further provides capabilities to decrypt local application databases to recover historical chat logs and user information. Additional functionality covers t

Open-deep-research is an autonomous research orchestrator and language model-based agent designed to execute multi-step investigations and automated web research workflows. It explores complex topics systematically by generating targeted search queries, processing multiple sources simultaneously, and recursively feeding synthesized findings back into search planning loops to expand its research scope. The system coordinates these tasks through a graph-based agent orchestration model that routes state between modular nodes for searching, reading, and reasoning. It accelerates data collection v

This is a collection of Python automation scripts and utility tools designed to handle repetitive technical tasks, system administration, and developer workflows. The project serves as a suite for task automation, data utility, and web automation. The collection includes specialized tools for multimedia processing, such as optical character recognition for extracting text from images, speech-to-text conversion, and real-time face and human body detection. It also features web scraping and monitoring capabilities to track product prices, fetch external API content, and automate interactions wi

This project is a vulnerability search engine and security knowledge base designed to collect and index public security disclosures. It functions as a vulnerability database crawler that extracts technical reports and security flaws from websites to create a searchable local archive. The system utilizes a security knowledge indexer and a full-text inverted index to convert unstructured crawled data into a structured format. This allows for keyword-based information retrieval, enabling the location of specific security flaws and technical details through a dedicated search interface. The plat

This is a collection of Python scripts designed for extracting data from popular Chinese websites and mobile applications. It functions as a multi-platform data extraction toolkit, capable of automating tasks such as downloading videos from platforms like Bilibili and Douyin, scraping product reviews and images from e-commerce sites like Taobao and JD.com, and booking train tickets on the 12306 railway system. The project distinguishes itself through its focus on automating specific, high-value tasks within the Chinese internet ecosystem. It includes capabilities for solving Chinese CAPTCHA c

franz-go is a low-level Go client library and wire protocol implementation for producing, consuming, and administering Kafka clusters. It functions as a zero-allocation network driver that utilizes a direct TCP communication layer to handle requests and responses. The project integrates a schema registry client for encoding and decoding structured data. It provides a programmatic interface for cluster administration, including the management of topics, access control lists, and broker configurations. The library covers data consumption through consumer groups, message production with transac

vllm-omni is a high-throughput serving engine and distributed inference framework designed for omni-modal models. It serves as a multi-modal model API server capable of generating text, image, video, and audio data, providing a standardized interface for remote client access. The system features a non-autoregressive generation engine for parallel media production and a robot policy inference server that acts as a real-time communication bridge to robotic hardware using specialized protocols. It supports hybrid execution models that combine sequential token generation with parallelized media g

Gitsome is a command-line interface tool for managing repositories, issues, and pull requests via the GitHub and GitHub Enterprise APIs. It provides an interactive shell environment featuring an embedded Python interpreter for programmatic automation and dynamic scripting, alongside a project bootstrapping utility for generating standard license and gitignore templates. The tool distinguishes itself with a schema-driven autocomplete engine that parses remote API definitions in real-time to provide context-aware command suggestions. It also implements an index-based navigation system that allo

PocketHub is an Android mobile client and Git client designed for managing GitHub repositories, issues, and pull requests. It functions as a mobile interface and API wrapper that interacts with the GitHub REST API to facilitate account and repository management on mobile devices. The application enables a mobile Git workflow, allowing users to track development progress and review code changes. This includes capabilities for GitHub repository administration and the management of user profiles and account activity. The project implements a mobile interface for interacting with GitHub via REST