layer_norm¶

dragon.nn.layer_norm(
  inputs,
  axis=- 1,
  epsilon=1e-05,
  **kwargs
)[source]¶

Apply the layer normalization. [Ba et.al, 2016]

The normalization is defined as:

\[y = \frac{x - \mathrm{E}[x]} {\sqrt{\mathrm{Var}[x] + \epsilon}} * \gamma + \beta \]

Parameters:

• inputs (Sequence[dragon.Tensor]) – The tensor x, gamma and beta.
• axis (int, optional, default=-1) – The start axis of normalized dimensions.
• epsilon (float, optional, default=1e-5) – The value to \(\epsilon\).

Returns:

dragon.Tensor – The output tensor.