Awesome GitHub RepositoriesWeight Pruning

PocketFlow is an integrated toolkit for deep learning model compression, distributed training, and mobile format optimization. It provides a system for reducing the size and complexity of neural networks to improve inference efficiency, featuring a dedicated engine for knowledge distillation and a mobile model optimizer. The framework differentiates itself through an automated hyperparameter tuning system that uses reinforcement learning and statistical models to determine optimal compression ratios and layer-wise bit allocation. It also includes a distributed training system that utilizes mu

Francesco Croce\ (University of Tübingen), Maksym Andriushchenko\ (EPFL), Vikash Sehwag\ (Princeton University), Nicolas Flammarion (EPFL), Mung Chiang (Purdue University), Prateek Mittal (Princeton University), Matthias Hein (University of Tübingen)

This codebase was developed by Jonathan Frankle and David Bieber at Google during the summer of 2018.

March 15, 2019: for our most updated work on model compression and acceleration, please reference:

A PyTorch implementation of this paper.

Hattie Zhou, Janice Lan, Rosanne Liu, Jason Yosinski

Code for the NeurIPS 2022 paper "Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning".

This repository contains code for the paper SNIP: Single-shot Network Pruning based on Connection Sensitivity (ICLR 2019).

Prune DNN using Alternating Direction Method of Multipliers (ADMM)

Adversarial Robustness vs Model Compression, or Both?

Repository with code to reproduce the results and checkpoints for compressed networks in our paper on novel pruning techniques with robust training. This repository supports all four robust training objectives: iterative adversarial training, randomized smoothing, MixTrain, and CROWN-IBP.

This is the official implementation of the paper: "Layerwise Sparsity for Magnitude-based Pruning", ICLR 2021.

This method identifies a binary weight or binary weight and activation subnetwork within a randomly initialized network that achieves performance comparable to, and sometimes better than, a weight-optimized network. The resulting binarized and pruned networks that achieve comparable performance…

This is an PyTorch implement of the paper ``Prune Your Model Before Distill It''.

This repository contains the codes for the following NeurIPS-2019 paper

This repository contains the code for reproducing the results in the following paper:

Code for the paper: ICLR 2022 Sparsity Winning Twice: Better Robust Generalization From More Efficient Training

This repository contains source code for the ICLR 2022 paper How many degrees of freedom do we need to train deep networks: a loss landscape perspective by Brett W. Larsen, Sanislav Fort, Nic Becker, and Surya Ganguli (arXiv version).

Code used for paper: ICLR 2022 Learning Pruning-Friendly Networks via Frank-Wolfe: One-Shot, Any-Sparsity, and No Retraining.

This repository is for our ICLR'22 paper: