1 个仓库
Awesome GitHub RepositoriesOptimizer State Compression
Memory-reduction techniques using low-rank approximations of momentum matrices, such as AdaFactor.
Distinct from Training Memory Optimizers: Focuses specifically on the optimizer's internal state compression for memory efficiency, distinct from general gradient checkpointing.
Explore 1 awesome GitHub repository matching data & databases · Optimizer State Compression. Refine with filters or upvote what's useful.
Awesome Optimizer State Compression GitHub Repositories
用 AI 发现最棒的仓库。我们将通过 AI 为您搜索最匹配的仓库。
- datawhalechina/so-large-lm
datawhalechina/so-large-lm
7,400在 GitHub 上查看This project is a comprehensive educational curriculum and structured learning path covering the full lifecycle of large language models. It provides a guided progression through the theory, architecture, training, and deployment of these models. The curriculum includes specialized guides on transformer architecture, model training tutorials, and frameworks for designing autonomous agents. It also provides dedicated resources for studying model safety and ethics. The material covers a wide range of technical capabilities, including distributed training strategies, parameter-efficient fine-tu
Details the use of AdaFactor to reduce training memory via low-rank momentum approximations.
