brightmart/text_classification
Text Classification
This project is a deep learning text classification framework and neural text analysis library. It provides tools for categorizing textual data, adapting large language models through fine-tuning, and treating classification tasks as sequence generation problems using transformer architectures.
The framework distinguishes itself through the implementation of ensemble learning, using boosting to combine predictions from multiple architectures to increase accuracy. It also includes a toolkit for fine-tuning pre-trained models via layer updates and the ability to restore model sessions for real-time online predictions.
The library covers a broad range of capabilities, including document hierarchy capture via attention mechanisms, convolutional feature extraction for n-grams, and multi-label categorization. It further supports temporal state modeling using episodic memory networks for transitive inference and contextual question answering.
Features
- Text Classification - Provides a comprehensive framework for assigning categories or labels to textual data using neural networks.
- Neural Text Analysis Libraries - Provides a library for extracting convolutional features and capturing document hierarchy using attention mechanisms.
- Sequence Generation - Treats text classification as a generation problem by producing token sequences using transformer architectures.
- Text Classification Frameworks - Ships a comprehensive framework of architectures and tools for categorizing textual data and generating token sequences.
- Multi-Label Classifiers - Supports associating multiple overlapping categorical labels with a single document using specialized classifiers.
- Structural Hierarchy Analysis - Implements attention mechanisms and bidirectional units to identify significant words and sentences within long documents.
- Document Hierarchy Modeling - Implements bidirectional units and attention mechanisms to identify the most significant words and sentences within long documents.
- Ensemble Learning - Combines multiple deep learning architectures through boosting and stacking to increase overall classification accuracy.
- Stacking Ensembles - Uses a layered stacking architecture where base model predictions serve as input for meta-models to increase accuracy.
- Episodic Memory Networks - Tracks the state of a story using episodic memory and gated mechanisms to perform transitive inference.
- Convolutional Feature Extractors - Uses one-dimensional convolution kernels and max pooling to capture local n-gram features for classification.
- Episodic Memory Networks - Tracks temporal state and story progress using gated mechanisms to perform transitive inference across long sequences.
- Partial Layer Fine-Tunings - Adapts large pre-trained models by updating only the classifier layer to align with specific datasets.
- Large Language Model Fine-Tuning - Adapts large language models to specific tasks by updating the classifier layer with task-specific data.
- LLM Fine-Tuning Toolsets - Provides a set of tools for adapting pre-trained large language models via targeted layer updates.
- Boosting Algorithms - Combines predictions from different architectures using boosting techniques to improve overall classification accuracy.
- Word Embeddings - Integrates external pre-trained word vectors to initialize model representations for improved semantic understanding.
- Generative Classification Models - Uses transformer architectures to treat classification tasks as sequence generation problems.
- Sequence-to-Sequence Transformer Architectures - Treats classification tasks as a generation problem by producing token sequences using a transformer encoder-decoder architecture.
- Embedding Layer Initialization - Provides the ability to load external pre-trained word vectors to initialize the semantic space of the model.
