Hyperparameter Tuning

The importance of hyperparameters

1. alpha

2. beta, number of hidden units, mini batch size

3. number of layers, learning rate decay

4. beta1, beta2, epsilon

Tuning process

1.Try random values. Don't use a grid 2.Use coarse to fine sampling scheme