Collective Operations

Scatter

Scatter tensor on rank 0 to all ranks

# Input
rank0 = tensor([0., 1, 2, 3])
# Output
rank0 = tensor([0.])
rank1 = tensor([1.])
rank2 = tensor([2.])
rank3 = tensor([3.])

Note: stepping stone to understanding reduce-scatter