awesome-repositories.comBlog

© 2026 Bringes Technology SRL·VAT RO45896025·hello@awesome-repositories.com

MCP Blog Curated searches Sitemap Privacy Terms

Datatrove | Awesome Repository

huggingfacedatatrove

0

View on GitHub↗

3,092 stars·273 forks·Python·Apache-2.0·0 views

Datatrove

Features

Data Curation and Filtering - Library for building scalable, platform-agnostic text processing pipelines.
Data Pipelines - Processes, filters, and deduplicates large-scale text data.
Data Processing - Platform-agnostic pipeline blocks for data processing.

AI search

Explore more awesome repositories

Describe what you need in plain English — the AI ranks thousands of curated open-source projects by relevance.

Start searching with AI

Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.