How this analysis was created: This summary and feature list were written by an AI model that read the project's README and public documentation pages. Each feature links to the documentation it came from; stars, license and language come straight from the GitHub API. The model does not read the source code, and the analysis is refreshed when the project is re-analysed. Learn more on our About page.
GeneralNewsExtractor is a specialized system for identifying and extracting structured news data through configurable HTML filtering and XPath-based parsing. It functions as an automated news scraper and web page content parser that isolates article bodies and metadata from news webpages. The system uses YAML or JSON configuration files to manage extraction rules and XPath definitions, decoupling the processing logic from site-specific selectors. This allows for targeted noise removal and the isolation of titles, authors, and publication dates through customizable filtering. The tool covers
Wechatsync is a multi-platform content synchronizer and cross-platform publishing tool. It extracts articles from webpages and distributes them to multiple social media and blogging platforms simultaneously. The system utilizes a web content extractor with reader-mode logic to strip advertisements and navigation elements from source pages. The project employs a markdown content pipeline that converts extracted web content into a standardized format for editing before redistribution. It features an automated media migrator that performs host-to-host image migration, downloading images from sou
Defuddle is a command line web parser and content extractor designed to isolate the primary article body from web pages and convert the result into standardized markdown. It functions as a content cleaner that removes layout clutter, such as sidebars and headers, to retrieve the main text and associated metadata. The tool provides a terminal interface that processes content from remote URLs, local files, or piped HTML streams. It supports custom content targeting, allowing users to specify CSS selectors to manually define the main content area when automatic detection is insufficient. The sy
Readability is a JavaScript library designed for web content extraction. It functions as a DOM parsing utility and article metadata extractor that isolates the primary text of a webpage by removing clutter such as advertisements and navigation bars. The library employs a heuristic-based content detector to predict if a webpage contains a parseable article before performing full extraction. It uses a parsing workflow to convert complex HTML documents into a simplified format, facilitating the implementation of distraction-free reader views. The tool covers several capability areas, including
python-goose is a Python library for web scraping and content extraction. It functions as an HTML boilerplate remover and article parser designed to isolate primary text and metadata from web pages by stripping away navigation, layout noise, and non-essential elements.
The main features of grangier/python-goose are: Web Article Extraction, Multilingual Text Processing, Stop-Word Text Scoring, Content Extractors, Web Content Noise Reduction, Web Article Metadata Extractors, Web Content Extraction Utilities, Web Media Extractors.
Open-source alternatives to grangier/python-goose include: generalnewsextractor/generalnewsextractor — GeneralNewsExtractor is a specialized system for identifying and extracting structured news data through configurable… wechatsync/wechatsync — Wechatsync is a multi-platform content synchronizer and cross-platform publishing tool. It extracts articles from… kepano/defuddle — Defuddle is a command line web parser and content extractor designed to isolate the primary article body from web… mozilla/readability — Readability is a JavaScript library designed for web content extraction. It functions as a DOM parsing utility and… esbatmop/mnbvc — MNBVC is a dataset pipeline and toolkit designed for the collection, cleaning, and normalization of massive text and… postlight/parser — Postlight Parser is a command-line tool that extracts the main article content from any web page URL, returning clean…