Wide expert parallelism
Named by 2 kept items in this workspace.
Search every workspace for Wide expert parallelism. It matches the name as written, so it can find other things that share it.
The items that name it 2
wideEP
In MoE models, places few experts on each GPU in one scale-up world so each expert handles larger token batches, at the cost of more all-to-all communication.
All techniques in this workspace.