SamsungSAILMontreal/TinyRecursiveModelsArchived

Features

• Latent State Recursive Refiners - Implements a recursive framework where the model iteratively updates its internal latent state to refine reasoning accuracy.
• Latent State Refinement - Improves model predictions by updating an internal latent state across multiple steps before final output.
• State Trackers - Maintains a persistent hidden representation that evolves as the model progresses through iterative reasoning steps.
• Iterative Prediction Refiners - Provides a mechanism to refine model outputs by iteratively feeding predictions back into the network for successive corrections.
• Reasoning Model Training Suites - Provides a training suite for optimizing small neural networks to solve complex logical puzzles recursively.
• Synthetic Reasoning Data Generators - Generates structured training data by mapping logical puzzles and maze solutions into reasoning sequences.
• Efficient Neural Architectures - Employs small-scale, parameter-efficient neural network architectures designed to minimize computational overhead.
• Recursive Answer Refiners - Improves predicted outputs by iteratively updating internal states and answers through multiple reasoning steps.
• Reasoning Dataset Builders - Constructs structured training and evaluation datasets for logical puzzles and maze solving.
• Neural Network Training Frameworks - Provides a framework for building and optimizing small-scale neural networks on complex logical tasks.
• Hardware Training Acceleration - Implements hardware-level optimizations for the neural network training process using GPU and TPU acceleration.
• Parameter-Efficient Training Toolkits - Ships a trainer for optimizing small-scale models on difficult tasks using parameter-efficient techniques.
• Parameter-Efficient Tuning Techniques - Optimizes small-scale models using parameter-efficient techniques to reduce resource requirements.

Star history