Open-source alternatives to Wechat Article Exporter

This project provides a comprehensive framework for building, deploying, and orchestrating autonomous agents within a decentralized network. It serves as a collection of patterns and examples for developing intelligent software entities capable of performing complex tasks, making decisions, and interacting with other agents to achieve shared goals. The framework distinguishes itself through its focus on multi-agent orchestration and decentralized communication. It enables the coordination of specialized agent teams that collaborate on workflows through structured messaging protocols, allowing

ArchiveBox is a self-hosted web archiving system designed to capture and preserve permanent static copies of webpages, media, and PDFs on personal infrastructure. It functions as a digital content curator and personal web archive manager, allowing users to import URLs from bookmarks, RSS feeds, and browser history to create a centralized, searchable knowledge base. The project is distinguished by its ability to archive private, paywalled, or login-protected content using browser cookies and authenticated session persistence. It ensures long-term availability by saving pages in multiple concur

Wechatsync is a multi-platform content synchronizer and cross-platform publishing tool. It extracts articles from webpages and distributes them to multiple social media and blogging platforms simultaneously. The system utilizes a web content extractor with reader-mode logic to strip advertisements and navigation elements from source pages. The project employs a markdown content pipeline that converts extracted web content into a standardized format for editing before redistribution. It features an automated media migrator that performs host-to-host image migration, downloading images from sou

Crawlee-python is a web crawling framework for building scalable scrapers using Python. It serves as a comprehensive tool for web scraping automation, providing a system to extract structured data from websites using both lightweight HTTP requests and headless browser automation. The framework is distinguished by its anti-bot evasion capabilities, which include browser fingerprint impersonation and tiered proxy rotation to bypass detection systems and solve challenges such as Cloudflare. It also incorporates artificial intelligence for autonomous website navigation and schema-based data extra

gpt-crawler is a web scraping utility designed to extract website content and convert it into structured text files for use as AI model knowledge bases. It functions as a data generator that crawls specified web addresses to produce the knowledge files required for building custom GPTs, grounding large language models, and providing context to AI agents. The system transforms raw HTML into clean Markdown text to reduce token usage and improve readability for AI models. It utilizes token-aware content chunking and output file size limitations to ensure generated datasets remain compatible with

This project is a collection of Python scripts and tools designed for web scraping, browser automation, and large-scale data extraction. It provides a set of implementations for retrieving information from websites and private APIs, including tools for multimedia downloading and social media data archiving. The toolset includes specialized mechanisms for bypassing anti-scraping measures through IP proxy pool rotation and multi-threaded crawlers. It also features capabilities for simulating browser sessions to handle authentication, intercepting session cookies, and decrypting network payloads

This project is a public community data API that provides a RESTful interface for accessing real-time discussions and metadata. It serves as a JSON data provider, offering structured community information for use in third-party client and server applications. The interface enables the retrieval of public items, stories, and comments, as well as public account information for specific users. It provides capabilities for fetching filtered lists of stories based on ranking or novelty and allows for the discovery of recent items to track new content updates. The system covers a broad range of da

This project is a community-curated directory of open-source software designed for deployment in private server environments and home labs. It serves as a comprehensive resource for discovering independent, self-hosted alternatives to mainstream cloud services, enabling users to maintain full data ownership and control over their digital infrastructure. The directory is structured through a hierarchical taxonomy that organizes a vast collection of applications into logical categories, ranging from media management and data analytics to private communication and team productivity tools. It dis

X-crawl is a Node.js-based web scraping framework designed to automate data collection from both static and dynamic websites. It integrates artificial intelligence to perform semantic parsing, allowing it to transform unstructured HTML into structured data formats that remain accurate even when website layouts or class names change. The project distinguishes itself through a comprehensive suite of stealth and reliability features. It manages crawler identity by randomizing device fingerprints and rotating proxy servers to bypass access restrictions. To handle complex, JavaScript-heavy interfa

This is a tool for downloading videos, images, and audio from the Douyin social media platform using shareable URLs or profile links. It can download individual posts, entire user profiles including all posts and liked content, collections, and music tracks, with options for watermark-free and high-quality output. The tool also supports live stream recording, comment collection, and keyword-based content search with JSONL export. The project distinguishes itself through an integrated REST API server that accepts download and transcription requests, tracks job status, and exposes health check

This project is an Amazon web scraper and e-commerce data extractor designed to retrieve product names, prices, and ratings. It functions as a headless browser crawler that converts unstructured web content from product listings into structured JSON and CSV formats. The tool incorporates anti-bot bypass capabilities to circumvent CAPTCHAs and security challenges. It achieves this through the use of residential proxy integration, automatic proxy rotation, and the modification of browser fingerprints to simulate human interaction patterns. The system provides broad web scraping capabilities, i

Wallabag is a self-hosted, open-source bookmark manager designed to archive web content for later reading. It functions as a personal knowledge management tool, allowing users to collect, store, and organize web pages into a centralized, searchable library. The platform provides a distraction-free reading experience by extracting the primary text and images from web pages while removing advertisements and navigation menus. This process ensures that saved articles remain accessible for offline reading, preserving the content even if the original source is removed from the internet. The system

Newspaper is a Python library designed for scraping, parsing, and analyzing web-based information. It functions as a framework for automated news aggregation and large-scale web content extraction, providing tools to download, clean, and structure text, metadata, and media from diverse online sources. The project distinguishes itself through a pipeline-oriented architecture that combines heuristic-based content extraction with natural language processing. It automatically identifies and isolates article bodies from web page boilerplate while simultaneously performing language detection, keywo

Reader is an AI data ingestion pipeline and web content parser designed to convert websites and documents into clean markdown for use with large language models. It functions as a headless browser content extractor and web-to-markdown converter, transforming URLs and PDF files into structured text formats while removing irrelevant web clutter. The system optimizes retrieval augmented generation by acting as a search optimizer that retrieves web results and applies re-ranking to improve context relevance. It further enhances content accessibility by using vision models to generate descriptive

node-crawler is a programmable web crawler for Node.js that manages request queues and automates data extraction. It functions as a rate-limited HTTP client and a headless HTML parser, providing the infrastructure to visit large sets of URLs asynchronously while preventing duplicate processing through task deduplication. The project distinguishes itself through a proxy rotation manager that cycles user agents and proxy servers to bypass access restrictions. It utilizes the HTTP/2 protocol to improve request performance and server compatibility during large-scale scraping operations. The syst

Discord Chat Exporter is a tool for extracting messages and media from Discord channels and direct messages into offline files. It functions as a backup utility and archival tool, using authentication tokens to retrieve chat history and metadata for long-term storage or history recovery. The system converts API data into readable documents and supports multi-format export options, including HTML, TXT, CSV, and JSON. It includes capabilities for automated chat backups by creating recurring tasks through the host operating system's task scheduler. The tool provides data management features suc

Lux is a command line video downloader written in Go designed for extracting and saving video and audio from various websites. It functions as a concurrent media downloader that increases transfer speeds by splitting files into fragments and downloading them using multiple threads. The tool serves as a playlist download manager capable of retrieving entire video collections or specific ranges of items. It also operates as a proxy-enabled media client, supporting HTTP and SOCKS5 proxies and session cookies to access region-locked, private, or age-gated content. Additional capabilities include

Trafilatura is a Python library and command-line tool for extracting clean, structured text and metadata from web pages. It downloads HTML content, identifies the main body of text, and strips away navigation, ads, and other boilerplate, returning the core article content along with fields like title, author, date, and URL. The tool can also extract user comments and test whether a page contains extractable text, making it a general-purpose web text extraction library. What distinguishes Trafilatura from simpler extractors is its configurable extraction pipeline, which offers high-speed, high

Piped is a privacy-focused video streaming service and self-hosted media proxy. It allows users to watch video and audio content without advertisements, user tracking, or the requirement of official accounts. The project utilizes a decentralized server network to distribute workloads and rotate outbound IP addresses, which helps bypass regional content restrictions and prevent provider blocks. It includes the ability to identify and skip sponsored segments within media files for a cleaner viewing experience. The service provides a JSON API for third-party integration to fetch video streams,

FreshRSS is an open-source, self-hosted web feed aggregator designed to collect, organize, and display content from multiple websites in a single, centralized interface. It functions as a comprehensive reader for standard syndication formats, allowing users to track updates from various sources while maintaining full control over their data and privacy. The platform supports multi-user environments, enabling individual account management and personalized reading experiences. The application distinguishes itself through its robust synchronization and extensibility capabilities. It provides a s

This project is a comprehensive framework for building and managing autonomous agent systems. It provides a unified architecture for orchestrating multi-agent societies, where specialized agents collaborate through roleplay to decompose and solve complex tasks. The system integrates language models with external environments, enabling agents to perform real-world actions through a standardized tool-calling abstraction layer. The framework distinguishes itself through its focus on iterative reasoning and data reliability. It employs automated feedback loops to refine agent outputs and self-eva

Kotaemon is an orchestration framework designed for building modular, agentic workflows that integrate document processing, retrieval-augmented generation, and multi-step reasoning. It provides a comprehensive platform for developing document-based question answering systems, allowing users to chain language models, prompt templates, and external tools into complex, automated pipelines. The system distinguishes itself through a highly modular architecture that emphasizes component-based composition and schema-driven data exchange. It supports autonomous agents capable of decomposing complex q

ai-goofish-monitor is an AI-driven marketplace monitor and containerized web scraper designed to track online listings. It uses multimodal large language models and natural language prompts to analyze product text and images, determining if items meet specific requirements. The system employs an anti-detection workflow that rotates network proxies and authenticated accounts to bypass rate limits. It captures browser cookies and session states to mimic real user behavior during automated requests. The project includes a task scheduler using cron expressions and an embedded SQLite database for

This project is a cross-platform desktop email client built with web technologies. It serves as an extensible mail application that allows users to manage and organize email correspondence and can link to self-hosted synchronization engines to manage user data. The application is designed for extensibility through a plugin architecture and logic extension hooks, enabling the addition of custom features such as text translation, email templating, and external service integrations. It further distinguishes itself by providing a customizable interface that supports user-defined CSS styling and t

This project is a cross-platform messaging SDK and client development library used to build custom Telegram applications. It functions as a comprehensive framework that manages network encryption, local data storage, and API communication, providing a C-compatible JSON interface that allows integration with any programming language. The library distinguishes itself by providing a full database manager for encrypted local caching and synchronized state, alongside a dedicated bot framework for creating interactive bots with business account integration. It enables the implementation of speciali

Gotify is a self-hosted notification server designed to centralize the receipt and dispatch of real-time messages. It provides a RESTful API and a WebSocket gateway, allowing users to programmatically send alerts and push notifications to connected clients. By maintaining a private infrastructure, the platform ensures full control over message history, data retention, and access management. The system distinguishes itself through a modular, plugin-based architecture that allows for the extension of core functionality, including custom HTTP endpoints and webhook event processing. It supports g

Gofr is a comprehensive framework for building production-ready microservices in Go. It provides a unified toolkit for developing RESTful APIs and gRPC services, offering built-in support for observability, database management, and distributed system communication. The framework distinguishes itself through its focus on developer productivity and system resilience. It automates common backend tasks such as CRUD handler generation, schema-driven code creation, and database migration orchestration, while preventing race conditions in clustered environments. To maintain stability, it includes in