dragon.distributed¶

Functions¶

all_reduce(…) : Reduce the input across all nodes in a group.

broadcast(…) : Broadcast the input from root node in a group.

is_initialized(…) : Return whether the distributed environment is initialized.

is_mpi_available(…) : Return whether the MPI backend is available.

is_nccl_available(…) : Return whether the NCCL backend is available.

get_backend(…) : Return the backend of given process group.

get_group(…) : Return the current default process group.

get_rank(…) : Return the rank of current process.

get_world_size(…) : Return the world size of environment.

new_group(…) : Create a new communication group.