1 repo
Programming abstractions that define how tokenization processes interact with larger machine learning pipelines.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Tokenization Interfaces. Refine with filters or upvote what's useful.
Transformers is a comprehensive library for machine learning that provides a unified interface for training, fine-tuning, and deploying transformer-based models. It supports a wide range of tasks, including text classification, language modeling, question answering, and sequence-to-sequence translation, while offering