all_reduce¶

dragon.vm.torch.distributed.all_reduce(
  tensor,
  op='sum',
  group=None
)[source]¶

Reduce the tensor across all nodes in a group.

Parameters:

• tensor (dragon.vm.torch.Tensor) – The tensor to reduce.
• op (str, optional) – The reduction op.
• group (ProcessGroup, optional) – The group for communication.

Returns:

dragon.vm.torch.Tensor – The output tensor.