Pipeline Parallelism
Use of Pipelining for GPUs to do training.
https://colossalai.org/docs/concepts/paradigms_of_parallelism/
GPipe paper:
Use of Pipelining for GPUs to do training.
https://colossalai.org/docs/concepts/paradigms_of_parallelism/
GPipe paper: