1 repository
Measures how longer generated solutions become exponentially harder for models to memorize than shorter ones.
Distinct from Model Evaluation and Analysis: Distinct from Model Evaluation and Analysis: focuses specifically on measuring memorization patterns rather than general model performance metrics.
Explore 1 awesome GitHub repository matching artificial intelligence & ml · Memorization Analyzers. Refine with filters or upvote what's useful.
GPT-Neo is an open-source distributed training framework designed for scaling GPT-2 and GPT-3-style language models across multiple devices using mesh-tensorflow for model parallelism. It provides the infrastructure to train transformer-based language models with billions of parameters across distributed computing environments, making large-scale language model research accessible outside of proprietary systems. The framework supports training both autoregressive GPT-style models and masked language models like BERT or RoBERTa, with configurable masking strategies and token handling. It inclu
Measures how longer generated solutions become exponentially harder for models to memorize than shorter ones.