Projects sharing features with Ripme

Pup is a command line tool for extracting and filtering data from HTML documents using CSS selectors. It functions as a parser and selector engine that isolates specific elements based on tags, IDs, classes, and attributes. The project provides utilities for converting selected HTML nodes into plain text, attribute values, or structured JSON objects. It includes a markup formatter that corrects missing tags and applies consistent indentation to improve the readability of HTML documents. The tool handles the retrieval of text content and attributes through a CSS selector engine, supporting co

UltimaScraper is an automated content downloader and media scraper designed to capture images and videos from OnlyFans accounts and save them to local storage. It functions as a media archive manager that organizes large volumes of downloaded content into structured folder hierarchies. The system includes a webhook notifier that sends automated alerts to external URLs upon the completion of download tasks. It utilizes custom naming patterns and directory structuring to sort files by username and media type. The tool provides resource control through concurrent download limiting to manage net

Cheerio is an HTML and XML parsing library and server-side DOM implementation. It functions as a markup manipulation tool and CSS selector engine, allowing users to parse, query, and modify HTML or XML documents in non-browser environments. The project provides a DOM-like tree representation of markup strings, enabling programmatic addition, removal, and modification of elements and attributes. It features a prototype-based plugin system that allows the extension of core functionality by adding custom methods to the document prototype. The library covers a broad range of capabilities includi

PHP HTML Parser is a server-side programming library and DOM parser designed to ingest markup documents and structure them into a navigable parent-child object tree. The library provides a stream-based character lexer and a configurable rule engine that manages parser strictness, whitespace preservation, tag closing behavior, and character encoding detection during document ingestion. Content loading is handled through a pluggable retrieval layer that accepts local file paths, raw strings, and remote network URLs. Once loaded, documents can be queried using a chainable CSS selector engine th

VidBee is a self-hosted media download manager that wraps the yt-dlp engine to download videos and audio from over 1000 websites. It functions as both a desktop client and a Fastify-based web service, managing downloads through a persistent queue with pause, resume, retry, and real-time progress tracking. The application uses cookie-based authentication to access login-gated, age-restricted, or subscriber-only content by importing browser cookies or Netscape-format cookie files. The application distinguishes itself through automated download workflows, including RSS and Atom feed monitoring t

so-novel is a web novel downloader and scraping engine designed to extract structured text from websites and convert it into electronic book formats. It functions as a multi-interface content extractor, providing a shared backend accessible via a web-based management dashboard, a terminal user interface, and a command line interface. The system utilizes a rule-driven approach for data extraction, using CSS selectors and XPath rules defined in external configuration files to map web elements to specific data fields. To maintain access to content, it includes a proxy-routed request pipeline to

Crawlee-python is a web crawling framework for building scalable scrapers using Python. It serves as a comprehensive tool for web scraping automation, providing a system to extract structured data from websites using both lightweight HTTP requests and headless browser automation. The framework is distinguished by its anti-bot evasion capabilities, which include browser fingerprint impersonation and tiered proxy rotation to bypass detection systems and solve challenges such as Cloudflare. It also incorporates artificial intelligence for autonomous website navigation and schema-based data extra

This project is a web-based manga and novel downloader and multi-site web scraper designed to extract images and text from diverse media platforms. It functions as a digital media archiver and EPUB e-book generator, using a plugin-based crawler architecture with site-specific scripts to define how content is extracted from various international websites. The system distinguishes itself through authenticated web crawling, using browser cookie simulation to access restricted or member-only content. It includes specialized capabilities for digital comic archiving, which organizes image sequences

Venera is a multi-source content reader and aggregator that allows users to browse and download media from various remote websites and local files through a unified interface. It functions as a local-remote media manager, synchronizing online content with local storage to enable offline viewing. The project utilizes a JavaScript-based content parser and aggregator to scrape and parse data from external web sources. This system allows for the definition of custom data extraction rules using JavaScript to fetch and display content from external websites. The platform covers remote media manage

udemy-dl is a Python command-line tool and web content scraper designed to download Udemy course videos, subtitles, and supplementary materials for offline personal use. It functions as a course media archiver that authenticates via user credentials or cookies to retrieve restricted media and metadata. The utility distinguishes itself through batch media retrieval, allowing the sequential download of multiple courses from a list of URLs. It provides granular control over the archive process, including the ability to filter specific chapters or lectures and export direct download links to a fi

Hardseed is a command-line forum media scraper and content archiver. It extracts images and torrent seeds from internet forums in bulk, saving the media and associated metadata to local directories. The tool utilizes a proxy-enabled web scraping engine that rotates network traffic through multiple proxy servers and protocols to bypass rate limits and network restrictions. It includes a keyword-based media filter that matches user-defined strings within entry titles to include or exclude specific topics. The system manages data extraction through batch-processed thread iteration and range-lim

This project is a distributed scraping engine designed to extract business details, customer reviews, and lead information from Google Maps. It functions as a business scraper and data extractor that can be deployed as a permanent system or as on-demand serverless functions. The system utilizes a proxy-routed web crawler to manage request origins via SOCKS5, HTTP, and HTTPS proxies. To locate contact information, it includes an email extraction tool that recursively crawls business websites linked within map listings. The software supports coordinate-based radius searches for efficient data

UserScripts is a collection of JavaScript browser userscripts designed to modify website behavior and add custom functionality to web browsers. It serves as a multi-purpose toolset for web page content automation, web interface enhancement, and specialized web scraping and downloading. The project distinguishes itself through a wide range of specialized utilities, including a browser-based text transformer for character encoding and terminology mapping, and tools for bypassing content censorship. It provides advanced web scraping capabilities such as deciphering obfuscated download links, agg

SwiftSoup is a cross-platform HTML processing library for Swift that converts raw HTML or XML strings and files into a structured document object model. It provides the core infrastructure to parse web content into a traversable tree, enabling programmatic access to page elements across iOS, macOS, and Linux. The library features a CSS selector engine for data extraction and a whitelist-based sanitization system to remove unsafe tags and attributes from user-submitted content. It optimizes repetitive document queries through memoized query caching. The project covers DOM manipulation for upd

This project is a command-line video downloader and web media extractor written in Python. It is designed to retrieve video and audio streams from various hosting platforms for local storage or real-time streaming via standard output. The system utilizes a framework of custom extractor classes to handle different websites and allows for the development of new extractors to extend compatibility. It supports accessing restricted, private, or region-locked content through the use of session cookies, user-agent headers, and proxy server routing. Capabilities include media format selection based

Huginn is an open-source automation platform that functions as an event-driven task automator and webhook integration engine. It enables the creation of agents that monitor web data and automate tasks across various web services, operating as a self-hosted web scraper and JavaScript workflow orchestrator. The system uses a directed graph of event flows to route and transform data between external APIs. It differentiates itself by allowing custom JavaScript execution within workflows to modify data payloads and by integrating human-in-the-loop automation to insert manual judgment or data entry

This project is a reference guide and collection of implementation patterns for replacing legacy libraries with native JavaScript. It provides a vanilla JavaScript reference guide, a modern web API cookbook, and a DOM manipulation cheat sheet to help migrate frontend dependencies to standard browser APIs. The project focuses on mapping library functions to native interfaces for DOM manipulation, network requests, and event handling. It includes a utility pattern library for common tasks such as string cleaning, type checking, and element styling. The covered capabilities include DOM element

This project is a comprehensive educational guide and framework for building web scrapers using Python. It provides a course-based approach to data extraction, combining a Python crawler framework with tutorials on web reverse engineering and network traffic analysis. The project distinguishes itself by covering advanced extraction challenges, including the decryption of obfuscated JavaScript and the bypass of anti-scraping measures. It specifically addresses mobile application scraping through the simulation of user interactions and the interception of network traffic. The capability surfac

This project is a curated library of Python code examples, educational resources, and programming tutorials. It functions as an educational repository designed to teach Python language fundamentals through practical implementation tasks, real-world exercises, and functional code snippets. The collection covers a diverse range of implementation examples, including the development of interactive websites and message boards using web frameworks. It also features scripts for audio speech processing, automated media processing for images, and the extraction of data from web content. Additional ca

Twikit is a Python library and API wrapper designed for interacting with X (Twitter). It simulates browser requests and mimics private network traffic to enable programmatic access to the platform without requiring an official API key. The project focuses on social media automation and data extraction, featuring tools for scraping user profiles, trending topics, and chronological tweet histories. It includes a session manager that handles user authentication, two-factor authentication, and cookie persistence to maintain active account access. The library's capabilities cover a broad range of

Requests is a Python HTTP client library used for sending HTTP requests and handling responses. It serves as a network client providing fundamental components for session management, proxy routing, multi-part uploading, and SSL/TLS certificate verification. The project distinguishes itself through a session manager that maintains cookies and reuses TCP connections to improve network performance. It also includes a dedicated multi-part form uploader for transmitting binary data and an integrated SSL/TLS certificate verifier to ensure encrypted and trusted communication. The library covers a b

Instaloader is a Python library and command-line utility designed for the automated retrieval, archiving, and analysis of Instagram content. It provides a programmatic interface to fetch media, captions, and metadata from public or private profiles, hashtags, and stories, while maintaining persistent user sessions for authorized access. The tool distinguishes itself through robust archive management and traffic control mechanisms. It supports incremental synchronization, allowing users to resume interrupted downloads and update local collections without redundant requests. To ensure reliable

This project is a Python-based automation toolkit designed to manage programmatic authentication and session persistence across web services. It provides a framework for executing automated login sequences, including the handling of interactive security challenges such as QR code verification and captcha resolution. The toolkit distinguishes itself by simulating native mobile application environments, allowing for the execution of scripts that require specific device-level headers and behaviors. It also incorporates hook-based interception to monitor workflow states and manage exceptions duri

This project is a Node.js web scraping framework designed to automate data extraction through a programmatic workflow of requests, parsing, and document interaction. It functions as a headless web crawler, an HTTP request manager, and a DOM parser and extractor. The framework distinguishes itself by combining a JavaScript execution engine to interact with dynamic content and a hybrid selection system that utilizes both CSS and XPath selectors. It includes specialized middleware for proxy rotation and cookie-jar session management to maintain authenticated states and manage automated traffic.

tdl is a command-line tool for Telegram account automation, media management, and data archiving. It provides a programmatic interface for downloading and uploading files, forwarding messages, and exporting chat history, member lists, and media into structured JSON files. The project distinguishes itself through a session management system that isolates multiple account identities using unique namespaces. It features expression-based routing for messages and uploads, allowing users to direct content to specific destination chats using custom logic and dynamic caption generation. The tool cov

TagSpaces is an offline-first file tagging and organization platform that lets you manage local files with portable metadata stored directly in filenames or sidecar JSON files, eliminating the need for a central database. It functions as a full-text file search engine, a Kanban board file organizer, a local AI file assistant, an S3-compatible cloud file manager, and a web clipper and bookmark manager, all within a single application. The project distinguishes itself through a local-first architecture where all file operations, indexing, and AI processing run entirely on the device, with cloud

CodeIgniter is a PHP web framework built on the Model-View-Controller pattern, designed for building full-stack web applications. It provides a lightweight toolkit with minimal configuration, organizing application logic into controllers, models, and views for clean separation of concerns. The framework includes a fluent query builder for constructing SQL statements programmatically, PSR-4 autoloading with namespace mapping, and a service-based dependency injection container for managing shared class instances. The framework distinguishes itself through its comprehensive set of built-in tools