Awesome GitHub RepositoriesHTML Parsing Command Line Tools

Pup is a command line tool for extracting and filtering data from HTML documents using CSS selectors. It functions as a parser and selector engine that isolates specific elements based on tags, IDs, classes, and attributes. The project provides utilities for converting selected HTML nodes into plain text, attribute values, or structured JSON objects. It includes a markup formatter that corrects missing tags and applies consistent indentation to improve the readability of HTML documents. The tool handles the retrieval of text content and attributes through a CSS selector engine, supporting co

Postlight Parser is a command-line tool that extracts the main article content from any web page URL, returning clean structured data including the title, author, date, excerpt, and lead image while stripping away ads and clutter. It uses a readability-based heuristic that scores HTML elements on text density and structural cues to identify the article body, and can accept pre-fetched HTML strings directly for parsing instead of fetching the URL. The tool distinguishes itself through a modular architecture that supports domain-specific extractor overrides, allowing custom JavaScript modules t

html-to-markdown is a tool that transforms HTML content and websites into Markdown text. It functions as an HTML to Markdown converter and a command line processor for converting files or standard input streams into Markdown documents. The project is an extensible conversion engine that allows for custom element rendering and the registration of plugins to modify the transformation process. It includes a content filter that uses CSS selectors to include or exclude specific HTML elements during conversion. The system covers broad capability areas including link resolution for relative URLs, r

The validator repository contains an open-source markup validation engine that checks HTML, CSS, and SVG documents for conformance against official W3C web standards. It operates both as a command-line utility for batch processing local files and remote URLs, and as a standalone web server and HTTP API for browser-based and remote document conformance checking. The software utilizes a rule-based conformance engine and event-driven stream parsing to evaluate document structures, syntax trees, and web accessibility guidelines. It provides URL parsing and normalization utilities that validate i