| Abstract
|
<p>We present a simple trick to get … <p>We present a simple trick to get an approximate estimate of the weight decay parameter lambda. The method combines early stopping and weight decay, into the estimate lambda=parallel to del E(W(es))parallel to/parallel to 2W(es)parallel to, where W(es) is the set of weights at the early stopping point, and E(W) is the training data fit error. The estimate is demonstrated and compared to the standard cross-validation procedure for lambda selection on one synthetic and four real life data sets. The result is that lambda is as good an estimator for the optimal weight decay parameter value as the standard search estimate, but orders of magnitude quicker to compute. The results also show that weight decay can produce solutions that are significantly superior to committees of networks trained with early stop ping.</p>ks trained with early stop ping.</p>
|
| Author
|
Thorsteinn Rögnvaldsson +
|
| DOI
|
http://dx.doi.org/10.1007/3-540-49430-8_4 +
|
| Diva
|
http://hh.diva-portal.org/smash/record.jsf?searchId=1&pid=diva2:541008
|
| EndPage
|
92 +
|
| HostPublication
|
Neural Networks : Tricks of the Trade +
|
| PublicationType
|
Conference Paper +
|
| Publisher
|
Springer Berlin/Heidelberg +
|
| Series
|
Lecture Notes in Computer Science ; 1524 +
|
| StartPage
|
71 +
|
| Title
|
A simple trick for estimating the weight decay parameter +
|
| Year
|
1998 +
|
| Has queryThis property is a special property in this wiki.
|
Publications:A simple trick for estimating the weight decay parameter +
, Publications:A simple trick for estimating the weight decay parameter +
, Publications:A simple trick for estimating the weight decay parameter +
, Publications:A simple trick for estimating the weight decay parameter +
, Publications:A simple trick for estimating the weight decay parameter +
, Publications:A simple trick for estimating the weight decay parameter +
, Publications:A simple trick for estimating the weight decay parameter +
, Publications:A simple trick for estimating the weight decay parameter +
, Publications:A simple trick for estimating the weight decay parameter +
, Publications:A simple trick for estimating the weight decay parameter +
|
| Categories |
Publication +
|
| Modification dateThis property is a special property in this wiki.
|
2 June 2017 20:22:03 +
|
