ReduceScatter

https://tech.preferred.jp/en/blog/technologies-behind-distributed-deep-learning-allreduce/ is really good