Projects sharing features with SingleFile

This project is a community-curated directory of open-source software designed for deployment in private server environments and home labs. It serves as a comprehensive resource for discovering independent, self-hosted alternatives to mainstream cloud services, enabling users to maintain full data ownership and control over their digital infrastructure. The directory is structured through a hierarchical taxonomy that organizes a vast collection of applications into logical categories, ranging from media management and data analytics to private communication and team productivity tools. It dis

Trilium is a hierarchical personal knowledge base and digital garden tool designed to organize information into a tree of linked nodes. It functions as a JavaScript programmable wiki and a self-hosted note server, allowing users to maintain a private database of documents synchronized across multiple devices. The platform distinguishes itself through an embedded scripting engine for automating tasks and a REST API that exposes internal data and actions to external tools. Users can further extend the system by modifying the user interface layout and styling through a custom theme engine. The

Wallabag is a self-hosted, open-source bookmark manager designed to archive web content for later reading. It functions as a personal knowledge management tool, allowing users to collect, store, and organize web pages into a centralized, searchable library. The platform provides a distraction-free reading experience by extracting the primary text and images from web pages while removing advertisements and navigation menus. This process ensures that saved articles remain accessible for offline reading, preserving the content even if the original source is removed from the internet. The system

DVC is a data versioning tool and pipeline orchestrator designed to track large datasets and machine learning models. It functions as a system for managing large data artifacts by storing lightweight metadata in version control while keeping the actual binaries in a separate cache. The project serves as an experiment tracker and remote storage synchronizer, enabling the execution and comparison of machine learning iterations based on hyperparameters and performance metrics. It provides a bridge for pushing and pulling these large data artifacts between local environments and cloud or on-premi

Violentmonkey is a userscript manager extension and JavaScript injection framework. It allows users to install and manage custom scripts and CSS to modify the behavior and appearance of websites based on metadata-driven trigger conditions. The project functions as a cloud-synced script manager, backing up and synchronizing scripts across different devices using external cloud storage services. It also serves as a cross-origin request bridge, providing a communication layer that enables scripts to bypass the same-origin policy to fetch data. The tool covers a broad range of browser automation

Markdownload is a browser extension that functions as a markdown web clipper, converting webpages and selected text into clean markdown files for offline storage and archiving. It operates as a content extractor that isolates the main document from the page while removing navigation elements and advertisements. The tool includes a template generator for injecting dynamic front-matter and metadata into documents via user-defined placeholders. It also serves as a local media downloader that saves remote images to the filesystem and updates links to reference those local files. Additionally, it

This is a tool for searching, downloading, and archiving articles and engagement metadata from WeChat official accounts. It functions as a web-based content scraper and data exporter, allowing for the automated retrieval of social media content and the collection of performance metrics. The project distinguishes itself through a system that captures session credentials and authentication cookies from desktop clients via a local proxy to access private engagement data. It utilizes a concurrent proxy-pool fetching mechanism to download large volumes of content while avoiding rate limits, and it

ArchiveBox is a self-hosted archiving tool designed for personal digital preservation and research data management. It functions as an automated web preservation engine that monitors URL inputs from bookmarks, browser history, or manual entries to capture and store permanent, offline copies of web content. By utilizing headless browser automation, the system renders dynamic web pages to ensure that captured snapshots, PDFs, and media assets remain accurate and accessible even if the original source disappears. The project distinguishes itself through a modular extractor pipeline and a task-qu

Omnivore is an open-source, self-hostable read-it-later application designed to centralize web articles, newsletters, and digital documents into a personal library. It functions as a comprehensive content archiver that captures web pages and stores them locally, ensuring permanent access and readability regardless of internet connectivity. The platform distinguishes itself through an event-sourced synchronization engine that maintains a consistent state across multiple devices by replaying user actions. It utilizes a headless web scraping service to extract clean text and metadata from raw we

Scrapy is a comprehensive framework designed for automated web data extraction and large-scale crawling. It operates on an asynchronous, event-driven engine that manages non-blocking network requests and data processing tasks, allowing for the efficient retrieval of structured information from web documents using path-based selectors. The system distinguishes itself through a highly modular architecture that supports complex data collection workflows. Users can implement custom middleware and signal handlers to intercept and modify request flows, while a priority-based scheduler manages concu

Spider is a web-based platform designed for automated data extraction, providing a centralized framework to collect, process, and route structured information from websites. It functions as a comprehensive pipeline that manages the entire lifecycle of data gathering, from initial configuration to final storage in external databases or message queues. The platform distinguishes itself through a visual configuration interface that allows users to define extraction rules and manage scraping templates without writing custom code. It supports both static and dynamic content retrieval by integratin

requests-html is a Python HTML parsing library and web scraping framework. It functions as an asynchronous HTTP client and a JavaScript rendering engine designed to fetch and parse web pages for structured data extraction. The project integrates a headless browser to execute JavaScript, allowing it to retrieve dynamically generated content that standard HTML parsers cannot see. It provides tools for automated data extraction using CSS selectors and XPath expressions to isolate specific text or attributes from HTML structures. The framework covers network operations including asynchronous pag

so-novel is a web novel downloader and scraping engine designed to extract structured text from websites and convert it into electronic book formats. It functions as a multi-interface content extractor, providing a shared backend accessible via a web-based management dashboard, a terminal user interface, and a command line interface. The system utilizes a rule-driven approach for data extraction, using CSS selectors and XPath rules defined in external configuration files to map web elements to specific data fields. To maintain access to content, it includes a proxy-routed request pipeline to

pipet is a command-line tool that turns web scraping into a piped data flow through Unix filters. It provides a set of specialized scrapers — for CSS selector extraction, headless browser JavaScript rendering, JSON API querying, and change monitoring — each outputting structured data that can be transformed by chaining additional commands. The tool uses declarative selectors (CSS and JSON path expressions) to define what to extract, automatically follows pagination links to collect data across multiple pages, and serializes results into JSON, custom-delimited text, or rendered templates. It c

Joplin is an open-source, cross-platform note-taking application designed for secure, private knowledge management. It functions as a local-first productivity platform, maintaining a complete relational database on the user's device to ensure offline availability and high-performance data retrieval. The application prioritizes data sovereignty by implementing an end-to-end encryption layer, which secures all information locally with a master key before any synchronization occurs. The platform distinguishes itself through a delta-based synchronization engine that transmits only specific file c

Crawlab is a distributed web scraping platform designed to centralize the management, deployment, and execution of large-scale data extraction tasks. It functions as a control plane that orchestrates scraping scripts and automated workflows across multiple nodes, providing a unified environment for managing complex data collection operations. The platform distinguishes itself through a distributed architecture that coordinates worker nodes via a central master, utilizing real-time communication to maintain oversight of all active processes. It ensures operational consistency by isolating task

This project is a command-line synchronization client for OneDrive and SharePoint libraries on Linux. It functions as a synchronization engine that aligns local filesystems with cloud storage through bidirectional, unidirectional, or download-only workflows. The client supports headless authentication for servers without web browsers and can be deployed as a background service or within a containerized environment. It enables the management of multiple distinct cloud accounts on a single system and integrates with shared SharePoint sites and document libraries. The synchronization engine inc

This is a cloud storage client and synchronization service for Linux that maintains data parity between local directories and OneDrive. It functions as both a command line interface and a background service to provide automated file backups and cloud synchronization. The tool supports multi-account management by isolating separate cloud profiles into distinct local directories through instance-based configuration. It utilizes delta-based data transfers and resumable chunked uploads to ensure large files are transmitted efficiently and can recover from interrupted transfers. The service provi

gdrive is a command line interface client for managing files, folders, and permissions within Google Drive. It functions as a backup utility and synchronization tool, allowing users to interact with their cloud storage through a terminal instead of a web browser. The project supports server-to-server integration by using service accounts and JSON credential files. This enables non-interactive identity verification and session establishment without requiring interactive user login prompts. The tool covers bidirectional and one-way synchronization between local directories and remote storage,

rclone is a command-line utility and sync engine for managing, synchronizing, and migrating files across numerous cloud storage providers. It functions as a storage management tool that enables bidirectional or one-way synchronization between local filesystems and remote cloud backends. The project acts as a unified cloud storage gateway, capable of merging multiple remote providers into a single directory tree. It further provides a filesystem mount to expose remote cloud storage as a local disk for direct operating system access and a transparent encryption wrapper to secure data before it

Nightmare is an Electron-based browser automation library and headless browser controller. It provides the infrastructure to programmatically navigate web pages, interact with DOM elements, and execute JavaScript within a background browser instance. The project distinguishes itself by integrating a full Chromium instance within an Electron shell, allowing for the management of browser sessions, network proxy settings, and persistent storage partitions. It enables the capture of page states as PNG screenshots, PDF documents, or HTML files. The tool covers a broad range of capabilities includ

Medium Unlimited is a browser extension and web content unblocker designed to remove membership restrictions and subscription limits from publishing platforms. It functions as a browser-based article unlocker that allows for the reading of premium content across various domains. The tool utilizes search engine integration to identify archived or mirrored versions of articles and automate redirects to full-text links. It combines these redirects with the ability to locate external publisher content when original articles are locked. The project achieves content access through document object

Avbook is a self-hosted, web-based digital library and catalog manager designed for organizing personal collections of adult videos and digital books. It serves as a centralized system for tracking media entries and associated metadata within a local database. The system functions as a media metadata aggregator and web-based scraper, automatically extracting video details and descriptive tags from external sources to populate the library. It also operates as a magnet link organizer, storing and indexing peer-to-peer file identifiers to simplify the discovery of downloadable media. The platfo

Huginn is an open-source automation platform that functions as an event-driven task automator and webhook integration engine. It enables the creation of agents that monitor web data and automate tasks across various web services, operating as a self-hosted web scraper and JavaScript workflow orchestrator. The system uses a directed graph of event flows to route and transform data between external APIs. It differentiates itself by allowing custom JavaScript execution within workflows to modify data payloads and by integrating human-in-the-loop automation to insert manual judgment or data entry

This project is a distributed scraping engine designed to extract business details, customer reviews, and lead information from Google Maps. It functions as a business scraper and data extractor that can be deployed as a permanent system or as on-demand serverless functions. The system utilizes a proxy-routed web crawler to manage request origins via SOCKS5, HTTP, and HTTPS proxies. To locate contact information, it includes an email extraction tool that recursively crawls business websites linked within map listings. The software supports coordinate-based radius searches for efficient data

Goutte is a PHP web scraper and DOM crawler designed for extracting data from websites. It functions as an HTTP client wrapper that enables the retrieval of web pages and the parsing of HTML content. The project provides a web form automator to programmatically fill and submit HTML forms to remote servers. It also includes a mechanism for automated website crawling by following links to discover and archive web content. The system supports stateful session management to maintain cookies and headers across requests. It further covers HTML data extraction through DOM-based element selection an