Sequence Parallelism
There seems like a few way to do this. This technique is a bit overloaded.
Learned from the huggingface tutorial.
There’s also Context Parallelism.
There seems like a few way to do this. This technique is a bit overloaded.
Learned from the huggingface tutorial.
There’s also Context Parallelism.