all_reduce

dragon.distributed.all_reduce(
  inputs,
  operation='MEAN',
  group=None,
  **kwargs
)[source]

Reduce the input across all nodes in a group.

Parameters:
  • inputs (dragon.Tensor) – The input tensor.
  • operation ({'MEAN', 'SUM'}, optional) – The reduce operation.
  • group (ProcessGroup, optional) – The group for communication.
Returns:

dragon.Tensor – The output tensor.