RMSprop¶
- class
dragon.optimizers.RMSprop(
lr=0.01,
momentum=0,
decay=0.9,
eps=1e-08,
**kwargs
)[source]¶ The optimizer to apply RMSprop algorithm. [Hinton et.al, 2013].
The RMSprop update is defined as:
\[\text{RMSprop}(g) = -m_{t} \\ \quad \\ \text{where} \quad \begin{cases} v_{t} = \text{decay} * v_{t-1} + (1 - \text{decay}) * g^{2} \\ m_{t} = \text{momentum} * m_{t-1} + \frac{\text{lr} * g}{\sqrt{v_{t}} + \epsilon} \end{cases} \]
__init__¶
RMSprop.__init__(
lr=0.01,
momentum=0,
decay=0.9,
eps=1e-08,
**kwargs
)[source]¶Create a
RMSPropoptimizer.- Parameters:
- lr (float, optional, default=0.01) – The initial value to \(\text{lr}\).
- momentum (float, optional, default=0) – The initial value to \(\text{momentum}\).
- decay (float, optional, default=0.9) – The initial value to \(\text{decay}\).
- eps (float, optional, default=1e-8) – The initial value to \(\epsilon\).
Methods¶
apply_gradients¶
Optimizer.apply_gradients(grads_and_vars)[source]Apply the gradients on variables.
- Parameters:
- grads_and_vars (Sequence[Sequence[dragon.Tensor]]) – The sequence of update pair.