Skip to content

gd-zhang/Weight-Decay

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

11 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Three Mechanisms of Weight Decay Regularization

This repo contains the official implementations of Three Mechanisms of Weight Decay Regularization.

  1. The config file for the experiments are under the directory of configs/.
  2. The modified optimization algorithms are in libs/.

Citation

To cite this work, please use

@inproceedings{
  zhang2018three,
  title={Three Mechanisms of Weight Decay Regularization},
  author={Guodong Zhang and Chaoqi Wang and Bowen Xu and Roger Grosse},
  booktitle={International Conference on Learning Representations},
  year={2019},
  url={https://openreview.net/forum?id=B1lz-3Rct7},
}

Requirements

This project uses Python 3.5.2. Before running the code, you have to install

How to run?

# example
$ python main.py --config configs/cifar100/resnet32/kfac/mb128_lr5e2_damp1e3_cov95_bn_aug.json

Credit

This repo uses a modified version of Tensorflow K-FAC.

Contact

If you have any questions or suggestions about the code or paper, please do not hesitate to contact with Guodong Zhang([email protected] or [email protected]) and Chaoqi Wang([email protected] or [email protected]).

About

Regularization, Neural Network Training Dynamics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages