3 个仓库
Awesome GitHub RepositoriesConvolutional Weight Standardization
Normalization techniques specifically for convolutional weights to improve training stability.
Distinct from Supervised Model Weight Optimization: Distinct from Supervised Model Weight Optimization: specifically addresses the standardization of weights within convolutional layers.
Explore 3 awesome GitHub repositories matching artificial intelligence & ml · Convolutional Weight Standardization. Refine with filters or upvote what's useful.
Awesome Convolutional Weight Standardization GitHub Repositories
用 AI 发现最棒的仓库。我们将通过 AI 为您搜索最匹配的仓库。
- mosaicml/composer
mosaicml/composer
5,485在 GitHub 上查看Composer 是一个 PyTorch 分布式训练框架，旨在实现大规模模型在多节点 GPU 集群上的扩展。它兼具大语言模型训练器、分布式模型优化器和训练生命周期管理器的功能。 该项目作为深度学习正则化库脱颖而出，提供诸如 Sharpness Aware Minimization、MixUp 和 CutMix 等专业优化技术，以提升模型的泛化能力。它还通过序列长度预热、渐进式层冻结以及用于大规模模型恢复的分片状态检查点技术，优化了训练流程。 该框架涵盖了广泛的功能领域，包括分布式训练编排、混合精度硬件管理和云原生数据流。它还为 GPU 内存诊断、训练发散检测和吞吐量跟踪提供了丰富的监控与可观测性工具。 该项目包含一个命令行启动器，可自动执行跨节点的分布式多 GPU 训练任务。
Normalizes convolutional weights in a model to improve training stability and convergence.
Python - flashlight/flashlight
flashlight/flashlight
5,443在 GitHub 上查看Flashlight is a standalone C++ machine learning library and tensor library used for building and training neural networks. It functions as a comprehensive neural network framework and automatic differentiation engine, providing the tools to construct computation graphs and calculate gradients via backpropagation. The project serves as a distributed training framework, utilizing all-reduce operations to synchronize gradients and parameters across multiple compute nodes and devices. It distinguishes itself through deep integration of high-performance tensor manipulation, native device memory in
Provides weight normalization wrappers for linear and convolutional modules to accelerate model convergence.
C++ - kmkolasinski/deep-learning-notes
kmkolasinski/deep-learning-notes
1,348在 GitHub 上查看This repository is an educational collection of implementations and research notes focused on deep learning architectures and optimization techniques. It provides modular code examples designed to demonstrate foundational and advanced concepts in machine learning, ranging from basic neural network structures to complex training strategies. The project distinguishes itself by offering practical implementations of specialized research methods, including capsule-based feature aggregation, gradient direction decoupling, and self-normalizing weight regularization. These materials allow for the stu
Implements weight normalization techniques to maintain statistical distributions and prevent vanishing gradients.
Jupyter Notebook
