⚠ Switch to EXCALIDRAW VIEW in the MORE OPTIONS menu of this document. ⚠ You can decompress Drawing data with the command palette: ‘Decompress current Excalidraw file’. For more info check in plugin settings under ‘Saving’
Excalidraw Data
Text Elements
PP0
PP1
PP2
PP3
PP4
TP4
incast
D0
D0
D0
D1
D2
D3
D0
D0
pxn transfer engine execution of one cpu numa
scheduler of each rank
D0
D0
D1,D2,D3
RDMA send D0
RDMA relay send
D0
D0
RDMA operation
Copy engine stream
desc ring[k]
doorbell signal
completed
status[k]
wait
Copy Engine
cuStreamWriteValue64 on copy stream （write doorbell signal, which in the shm） which make sure data is ready
dependency op
dependency op
pxn transfer engine
shm in dram
relay buffer in hbm（static size）
rdma task D0
rdma task D0
all intra ranks use the same shared memory
next CE task
time
D0 CE task
D0 CE task
D0 CE task
D0 CE task
Rail Resolver
same rail
cross rail
角色 A：CE sender
角色B：relay
thread1(poll cq)：ibv_poll_cq → slice→markSuccess() ◀─ CQ
thread2(post wr)：ibv_post_send ────────▶ NIC
thread4(monitor)：CQ 停滞诊断 ＋ heartbeat 快照 ＋ 超时 reap
wait credit
切 8MB 的 pieces (MPSC queue)
cudaMemcpyBatchAsync
cuStreamWriteValue64
lane0
lane1
lane0
lane1
lane0
lane1
update shm
lane0
credit=len(lane) completed
desc head++
lane[0][1][2][3][4][5][6][7]
producer
check doorbell signal
read desc
submitTransfer(inner_)
收割 getBatchTransferStatus
归还槽 + credit
①
②
③
⑤
⑥
⑦
④
thread3(post desc/cudaMemcpyBatch/doorbell)
consumer
①
⑥
PP0
PP1
PP2
PP3
sglang/vllm
sglang/vllm
sglang/vllm
sglang/vllm
transfe_data x threads x pp ranks
切 64KB slice → slice_queue_[shard][peer_nic_path]
⑥
⑥
⑥
①
①
⑥
⑥
①
head
head++
head++
head++
head++
completed
completed ++
completed ++
head - completed > credit
intranode
internode
one side
①
拿 slice
标记 slice 完成
144MB
144MB
144MB
144MB
144MB
144MB
8MB slots0
8MB slots1
8MB slots2
8MB slots3
…
8MB slots16
8MB slots17
PP4
PP5
PP6
PP7
copy time is hidden
with in the rdma
角色C：RDMA sender