⚠ Switch to EXCALIDRAW VIEW in the MORE OPTIONS menu of this document. ⚠ You can decompress Drawing data with the command palette: ‘Decompress current Excalidraw file’. For more info check in plugin settings under ‘Saving’
Excalidraw Data
Text Elements
Compute Stream
Communication Stream (A2A)
Stream
BASELINE(No Overlap)
F/Attention
F/Dispatch a2a x 2
F/MLP
F/Combine a2a x 1
B/Combine a2a x 1
B/MLP
B/Attention
Overlap waith W/D Split mlp
F/Attention
F/Dispatch a2a x 2
F/MLP
F/Combine a2a x 1
B/Combine a2a x 1
B/ DgradMLP
B/ WgradMLP
B/Dispatch a2a x 1
B/Dispatch a2a x 1
B/Attention
Speed Up
permute rank-expert major
permute rank-expert major
Overlap waith W/D Split mlp and permute
permute rank-expert major
permute rank-expert major