⚠ Switch to EXCALIDRAW VIEW in the MORE OPTIONS menu of this document. ⚠ You can decompress Drawing data with the command palette: ‘Decompress current Excalidraw file’. For more info check in plugin settings under ‘Saving’
Excalidraw Data
Text Elements
mooncake
connector
batch_transfer_sync_write
vllm
实例化TransferEngine()
initialize
get_rpc_port
batch_register_memory
openSegment
allocateBatchID
submitTransferWithNotify or submitTransfer
rdma
closeSegment
rdma
transport
RdmaTransport:: submitTransferTask
ibv_post_send
request1—n
slice1
slice2
slice3 …
lkey
dst_addr src_addr
lkey
dst_addr src_addr
lkey
dst_addr src_addr
WorkerPool:: submitPostSend()
slice1
slice2
slice3 …
lkey
dst_addr src_addr
lkey
dst_addr src_addr
lkey
dst_addr src_addr
slice1
slice2
slice3 …
lkey
dst_addr src_addr
lkey
dst_addr src_addr
lkey
dst_addr src_addr
rkey peer_nic retry_cnt
rkey peer_nic retry_cnt
rkey peer_nic retry_cnt
worker thread:: transferWorker 从 slice que 里面取出来一个 slice 然后走 ibv_post_send
getBatchTransferStatus
ibv_poll_cq
ibv_reg_mr
or
ibv_reg_dmabuf_mr
每个 engine 完成各自整块 kv cache的 mr 地址范围，rkey，NIC GID 等注册，在 host:port 等对端 的RPC 服务来取
TransferEngineImpl:: getRpcPort
返回 engine 监听的 RPC 端口
TransferEngineImpl()
scheduler
process
worker
process
xxxConnectorWorker
xxxConnectorScheduler
第一次会拿回 整个kv cache buffer 的地址范围和 rkey