Optimizer

class dragon.optimizers.Optimizer(
  scale=1,
  clip_norm=0,
  weight_decay=0,
  name=None
)[source]

The base class of optimizers.

__init__

Optimizer.__init__(
  scale=1,
  clip_norm=0,
  weight_decay=0,
  name=None
)[source]

Create a Optimizer.

Parameters:
  • scale (float, optional, default=1) – The scaling factor to gradient.
  • clip_norm (float, optional, default=0) – The maximum L2 norm to clip gradient.
  • weight_decay (float, optional, default=0) – The L2 penalty factor to weight.
  • name (str, optional) – The optional name for shared slots.

Methods

apply_gradients

Optimizer.apply_gradients(
  values_and_grads,
  lr_mult=None,
  decay_mult=None
)[source]

Apply the gradients on values.

Parameters:
  • values_and_grads (Sequence[Sequence[dragon.Tensor]]) – The values and grads.
  • lr_mult (number, optional) – The multiplier to learning rate.
  • decay_mult (number, optional) – The multiplier to weight decay.