Fully Sharded Data Parallel (FSDP) https://engineering.fb.com/2021/07/15/open-source/fsdp/ Related DDP