ericchiang/pup
Pup
Pup is a command line tool for extracting and filtering data from HTML documents using CSS selectors. It functions as a parser and selector engine that isolates specific elements based on tags, IDs, classes, and attributes.
The project provides utilities for converting selected HTML nodes into plain text, attribute values, or structured JSON objects. It includes a markup formatter that corrects missing tags and applies consistent indentation to improve the readability of HTML documents.
The tool handles the retrieval of text content and attributes through a CSS selector engine, supporting complex tags and combinators. It also manages character encoding through automatic detection or specified charsets to ensure correct text rendering during the extraction process.
Features
- CLI Data Extractors - Provides a command line interface for extracting specific elements from HTML using CSS selectors.
- CSS Selector Engines - Implements a CSS-style selector engine for isolating specific HTML elements based on tags, IDs, classes, and attributes.
- CSS Selector-Based Element Isolation - Supports complex CSS selectors and combinators to precisely isolate specific HTML elements.
- HTML Parsing - Provides a parser that constructs a DOM by automatically correcting malformed HTML markup.
- Selector-Based Element Isolation - Filters HTML documents to find specific elements based on tags, IDs, classes, and attributes.
- Command Line HTML Parsers - Provides a terminal-based tool for extracting specific data from HTML via CSS selectors.
- HTML Parsing Command Line Tools - Ships a command line interface for retrieving and filtering specific elements from HTML documents.
- CSS Selector Data Extractors - Uses precise CSS selectors to filter and collect specific elements from HTML documents.
- DOM Element Filtering - Implements a CSS selector engine to isolate specific HTML elements within a document.
- PDF and HTML Content Extraction - Extracts all text content from selected HTML nodes and their children.
- Web Scraping - Provides utilities for extracting structured data from websites and online sources using CSS selectors.
- HTML to Text JSON Converters - Converts selected HTML nodes into plain text, attribute values, or structured objects.
- HTML Formatters - Includes a markup formatter that corrects missing tags and applies consistent indentation to improve HTML readability.
- HTML Formatting - Controls the visual presentation of HTML data through indentation and colorization.
- Noise Reduction Filtering - Isolates specific HTML nodes based on tags and classes to filter out noise from webpages.
- DOM to JSON Serialization - Transforms selected HTML elements into machine-readable JSON objects for interoperability.
- Markup Cleaning - Includes a utility to fix missing tags and apply consistent indentation to HTML documents.
- HTML Data Attribute Extraction - Retrieves the values of specific attribute keys from all selected HTML nodes.
- Terminal Text Formatting - Applies indentation and character escaping to make HTML output readable in a terminal.
- HTML and XML Processing - Query HTML pages with CSS selectors.
