1 Repo
Awesome GitHub RepositoriesLanguage-Based Output Filters
Filters extracted text content to include only pages matching a specified language code.
Distinct from Content and Language Filtering: No candidate covers filtering extracted text by language; closest is Content and Language Filtering which applies to crawl queues, not extraction output.
Explore 1 awesome GitHub repository matching data & databases · Language-Based Output Filters. Refine with filters or upvote what's useful.
Awesome Language-Based Output Filters GitHub Repositories
Finde die besten Repos mit KI.Wir suchen mit KI nach den am besten passenden Repositories.
- adbar/trafilatura
adbar/trafilatura
5,319Auf GitHub ansehenTrafilatura is a Python library and command-line tool for extracting clean, structured text and metadata from web pages. It downloads HTML content, identifies the main body of text, and strips away navigation, ads, and other boilerplate, returning the core article content along with fields like title, author, date, and URL. The tool can also extract user comments and test whether a page contains extractable text, making it a general-purpose web text extraction library. What distinguishes Trafilatura from simpler extractors is its configurable extraction pipeline, which offers high-speed, high
Trafilatura restricts extracted output to a target language by identifying and filtering text via language codes.
Pythonarticle-extractorcorpus-buildercorpus-tools
