Projects sharing features with Fake Useragent

SpringSide 4 is an enterprise Java reference architecture and utility library built on the Spring Framework. It provides a pragmatic, best-practice application stack for building RESTful web services, web applications, and data access layers, along with a curated collection of high-performance utility classes for common operations like text, date, collection, reflection, concurrency, and I/O handling. The project distinguishes itself by combining a complete reference application scaffold with production-oriented infrastructure. It includes a JPA-based data access layer that automatically tran

This project is a PHP user agent parser and mobile device detector. It analyzes HTTP user agent strings to identify the visitor's web browser, operating system, and device type. The library provides a dedicated integration for the Laravel web framework. It distinguishes between mobile phones, tablets, and desktop computers while identifying web robots and the specific names of search engine crawlers. The tool's capabilities include extracting preferred languages from request headers and verifying specific user agent properties. It uses regular expression pattern matching and static data mapp

MechanicalSoup is a Python web automation library and scraping framework designed to simulate browser sessions and navigate websites without requiring JavaScript execution. It functions as an HTML parsing tool and HTTP session manager, allowing for the programmatic retrieval of page content and the automation of web interactions. The library distinguishes itself by combining session persistence with automated form interaction. It maps user data to HTML input fields and selection boxes for programmatic submission and maintains authenticated states by managing cookies and user-agent headers acr

Scraperr is a self-hosted web scraping and crawling platform designed for extracting structured data from websites using XPath selectors. It functions as a containerized system for managing scraping jobs through a queue and analyzing the resulting content using artificial intelligence. The project differentiates itself through its Kubernetes-native architecture, allowing for scalable deployment and management via package managers. It includes a crawling engine capable of domain-level spidering to discover linked pages and a data analyzer that uses artificial intelligence to query extracted we

X-Ray is a web scraping framework and asynchronous web crawler designed to extract structured data from websites. It functions as an HTML data extractor that transforms raw page content into a defined schema using CSS-style selectors. The project implements a headless browser crawler capable of executing JavaScript to render dynamic content. It handles website content discovery through a breadth-first crawling strategy and automatic pagination discovery to traverse multi-page result sets. The framework manages web data pipelines using a concurrency-limited request queue and request rate cont

snscrape is a Python-based social media web scraper and crawler designed to extract public posts, profiles, and hashtags from social networks without the use of official APIs. It functions as an archival tool and a utility for open-source intelligence data collection, allowing for the gathering of publicly available information to investigate trends and people. The tool facilitates social media data extraction for research and archival purposes, enabling the creation of historical records of conversations and user activity. It supports workflows for academic social analysis and the export of

bilibili-api is a Bilibili API wrapper and content scraper designed for programmatically accessing video metadata, user profiles, and content data. It functions as an anti-bot crawler framework and a WebSocket live chat client for retrieving platform information and real-time interaction data. The project incorporates tools to bypass anti-crawling measures and rate limits through the use of proxies and TLS fingerprint spoofing. It also includes logic for mapping and converting various video and content identifiers to ensure consistent data retrieval across different endpoints. Its capability

CrawlerTutorial is a comprehensive Python web scraping tutorial and framework designed for extracting data from static and dynamic websites. It functions as a web data extraction pipeline and an HTTP request orchestrator, covering the full lifecycle of scraping applications from initial fetching to final data storage. The project provides specialized guidance on anti-bot bypass techniques and web API reverse engineering. It includes methods for evading browser detection through identity masking and proxy rotation, as well as techniques for identifying hidden API endpoints by analyzing network

Stealth is a distributed automation engine designed for headless browser orchestration, web scraping, and decentralized network coordination. It functions as a peer-to-peer framework that enables multiple nodes to align automated tasks, share cached web resources, and route traffic through a decentralized proxy architecture. By abstracting the underlying browser implementation, the project provides a unified interface for executing navigation and data extraction workflows across diverse environments. The platform distinguishes itself through its ability to synchronize browser sessions and aut

Bowser is a browser detection library that parses user-agent strings to identify a browser's name, version, rendering engine, and operating system. It functions as a user-agent parser and version constraint checker, extracting structured browser and platform details from raw user-agent strings without external dependencies. The library distinguishes itself by integrating User-Agent Client Hints alongside traditional user-agent data for more accurate browser identification in modern environments. It provides cross-platform browser detection that works consistently across desktop and mobile ope

Scraperjs is a JavaScript web scraping library and headless browser automation tool designed to extract structured data from both static HTML and JavaScript-rendered websites. It functions as an asynchronous scraping workflow orchestrator and a selector-based engine for transforming unstructured web content into usable data. The library distinguishes itself by managing a shared browser instance across multiple tasks to optimize resource usage and reduce startup latency. It supports custom script injection into sandboxed browser environments to manipulate the DOM or execute browser-side logic

curl_cffi is a Python HTTP client built on libcurl that focuses on browser fingerprint impersonation to evade anti-bot detection. By replacing default TLS handshake and HTTP/2 settings with those extracted from real browsers like Chrome and Firefox, it allows HTTP requests that closely mimic actual browser traffic, reducing the likelihood of being blocked by services that fingerprint automated clients. Beyond fingerprint impersonation, curl_cffi offers a dual API supporting both synchronous and asynchronous execution, with per-request proxy assignment, automatic retry with exponential backoff

current-device is a JavaScript device detection library used to identify operating systems, device categories, and screen orientations within a web browser environment. It serves as a browser environment detector and client-side mapper that translates browser metadata and user-agent strings into predefined device labels and hardware categories. The library provides tools for executing specific JavaScript logic based on the detected mobile or desktop platform. It also functions as a conditional styling tool and document body class injector, applying descriptive HTML classes to enable device-sp

Colly is a web scraping framework and concurrent crawler written in Go. It provides a system for traversing web pages, following links, and extracting structured data from HTML and XML documents. The framework includes a distributed scraping engine designed to spread data collection tasks across multiple instances to increase throughput. It ensures compliance with website owner policies by automatically reading and respecting robots.txt files. The system manages request lifecycles through domain-based rate limiting, concurrency controls, and session management via a stateful cookie jar. It s

This repository is a collection of Java backend projects, encompassing a web application suite, a real-time chat server, a web crawler, and an authentication system. It provides implementations for building server-side logic and business rules to process data and handle web requests. The project includes a networked messaging system for instant bidirectional communication and an automated web scraping engine that parses HTML to aggregate data into local storage. It also features a security framework for managing user permissions and generating visual captchas to prevent automated bot access.

Obscura is a web scraping infrastructure and headless browser server designed for AI agents. It provides a system for AI models to control browser sessions, interact with websites, and extract web data using a WebSocket implementation of the Chrome DevTools Protocol. The project focuses on bot detection evasion by randomizing browser fingerprints, masking native functions, and blocking tracking scripts to mimic human behavior. It further secures identities through a traffic layer that routes network requests via HTTP or SOCKS5 proxies. The system supports large-scale data extraction through

Camoufox is a Firefox-based stealth automation browser designed to evade detection during automated browsing. It combines a fingerprint randomization engine that generates thousands of unique device attributes per session, native-level API interception to spoof WebRTC, WebGL, media, and other fingerprintable properties, and human behavior simulation that moves the cursor along natural, distance-aware trajectories. The browser is compiled from source with build-time stealth patches and runs headlessly via a lightweight virtual display buffer, making it suitable for web scraping, automated testi

python-goose is a Python library for web scraping and content extraction. It functions as an HTML boilerplate remover and article parser designed to isolate primary text and metadata from web pages by stripping away navigation, layout noise, and non-essential elements. The tool features multilingual processing capabilities, utilizing language-specific stop-word analyzers to identify and extract primary content across different languages. It also identifies and collects embedded media, including source URLs and embed codes for lead images and videos associated with an article. The library cov

Haxl is a Haskell library and remote service request orchestrator designed for coordinating concurrent data fetching, request batching, and caching across multiple remote service providers. It functions as a framework for retrieving data from external databases and web services while minimizing network round trips. The project distinguishes itself through an applicative-based request batching system that groups multiple individual data requests into single calls to reduce network overhead. It employs an asynchronous parallel request scheduler to execute independent requests concurrently and u

This project is a containerized search infrastructure designed to deploy a privacy-focused metasearch engine. It acts as a self-hosted search proxy that aggregates results from multiple external web, image, and academic search providers while anonymizing requests and stripping trackers to protect user identity. The system utilizes Docker to orchestrate the search instance, integrating caching mechanisms and reverse proxy support to ensure a private and efficient search environment. It employs a modular adapter-based integration to standardize diverse external API responses and a processing pi

uncaptcha2 is an audio captcha bypass tool and ReCaptcha solver designed to circumvent bot detection systems. It functions as a speech-to-text captcha solver that converts audio security challenges into text to automate the completion of digit-based verification steps. The tool enables automated web scraping by removing security barriers and bypassing ReCaptcha challenges. It facilitates automated form submission by programmatically solving the verification steps required to access restricted websites.

so-novel is a web novel downloader and scraping engine designed to extract structured text from websites and convert it into electronic book formats. It functions as a multi-interface content extractor, providing a shared backend accessible via a web-based management dashboard, a terminal user interface, and a command line interface. The system utilizes a rule-driven approach for data extraction, using CSS selectors and XPath rules defined in external configuration files to map web elements to specific data fields. To maintain access to content, it includes a proxy-routed request pipeline to

DeepDanbooru is a deep learning tool for tagging anime-style images with Danbooru-style tags. It uses a pre-trained convolutional neural network to analyze images and predict tags identifying characters, attributes, and artwork details. The project provides a complete pipeline for training custom tag recognition models. Users can prepare datasets by downloading tag definitions from a remote Danbooru server using authenticated API requests, then store image-tag pairs in a structured SQLite database. The training workflow supports filtering datasets by rating or score criteria, configuring hype

Isomorphic-git is a full implementation of the Git version control system written in JavaScript. It serves as a programmatic client library that allows developers to perform core version control operations without requiring a system-level Git binary installation. The library is designed as an isomorphic codebase, meaning it runs identically across both client-side web browser environments and server-side Node.js runtimes. It achieves this by using a filesystem-agnostic storage model and a pure JavaScript implementation of the Git core logic and the Smart HTTP protocol. The project covers a c

is.js is a JavaScript validation library and data type checker. It provides a suite of utilities to verify whether variables are primitives, arrays, functions, or specific object types. The project allows for the implementation of custom validation logic and the use of naming namespaces to override default rules and regular expressions. It covers a broad range of validation capabilities, including numeric arithmetic properties, date and time analysis for relative and absolute ranges, and string format verification for patterns such as emails, URLs, and IP addresses. The library also includes

JioNLP is a Chinese natural language processing toolkit designed for cleaning, normalizing, and extracting structured information from unstructured text. It functions as a linguistic analyzer for Chinese characters and a rule-based named entity extractor, providing a specialized system for sentiment scoring and synthetic data generation for machine learning workflows. The project features a lexicon-based sentiment analysis engine that computes numerical emotional tone scores and a data augmentation library that uses back-translation and synonym replacement to expand training datasets. It incl

This library is a JavaScript plugin that enhances standard text input fields with real-time, predictive search suggestions. It functions as a reusable interface component that monitors user input to display dynamic dropdown menus, allowing for the selection of data from either local arrays or remote server endpoints. The plugin distinguishes itself through its ability to manage asynchronous data retrieval, utilizing request orchestration and debouncing to optimize network traffic. It provides granular control over the suggestion lifecycle and rendering, allowing developers to define custom HT