optim
1. 双口网卡问题
背景: 在 b300 上验证发现，8 gpu 配 16 个 400g nic 时，会出现 redispatch(mooncake 的 fault tolerance逻辑)，而 pxn 我最开始假设为每个 gpu 用一个最近 pcie nic 来转发流量。当请求拿着tcp 传来的 target 的结构体后(preferred_hca字段内是 2 个 nic)，pxn 认为对端只有一个和我是可以发的 故流量进来 不知道选本地哪个网卡发 出现错误。
增加双口网卡支持，确定 pfc 得到有效解决，commit: 71b641d0de3c98f3616ce6e731d60915e4056a0d ✅ 2026-08-29
2. 每个 transfer engine的 pxn worker 效率低
背景：第一版设计时，我们的一个 pxn worker 需要兼顾把上层任务下来拆分后，丢到 relay buffer 内，还有负责 relay buffer 内的任务再丢给 rdma 搬走，还有对应这俩任务的结束时间的 pollcq和 update completed等等控制面操作，且每次扫一遍每个 lane，对每个 lane 内的 18 个 slots 都推进一格消耗一格，这么操作导致的任务提交和消耗都很慢。
sender 侧改为 2 个线程分别下 cudaMemcpyBatchAsync、submitToRdma，以及 poll 任务从一次 poll 一个改为 poll 一堆。性能确定达到 97%原生 ✅ 2026-08-29
3. 分析瓶颈(97→100%)
在日志内增加 lane 内每个 slots 状态，确定当前瓶颈在下nvlink操作被 sglang 阻塞。具体原因为多个 batch 操作会被 reshard 到上层的一个线程内，导致并发的 batch—WriteSync 操作少，然后下nvlink操作的并行度大打折扣。 ✅ 2026-08-30
发现 sglang 内单 decode 的时候即使每个 engine 有多个 thread 下 batch put 的操作，但是多线程的 reshard 是分给多个 decode 的。所以我调整为一个 d 的时候也能用多个 thread 下 transfer 的请求。（这个收益是明确为正的）。
/root/sglang/python/sglang/srt/disaggregation/mooncake/conn.py
diff --git a/python/sglang/srt/disaggregation/mooncake/conn.py b/python/sglang/srt/disaggregation/mooncake/conn.py
index 66ac63b34c..f1ac2acfee 100644
--- a/python/sglang/srt/disaggregation/mooncake/conn.py
+++ b/python/sglang/srt/disaggregation/mooncake/conn.py
@@ -2166,12 +2166,7 @@ class MooncakeKVManager(CommonKVManager):
# add further chunks into the transfer queue.
return
- # NOTE(shangming): sharding according to the dst_infos to make sure
- # requests with the same dst_sessions will be added into the same
- # queue, which enables early abort with failed sessions.
- dst_infos = self.transfer_infos[bootstrap_room].keys()
- session_port_sum = sum(int(session.rsplit(":", 1)[1]) for session in dst_infos)
- shard_idx = session_port_sum % len(self.transfer_queues)
+ shard_idx = bootstrap_room % len(self.transfer_queues)
if trace_ctx is None:
trace_ctx = TraceNullContext()
4. 为什么原生的 mooncake 测试时候发现 rank0的流量会比别人多
因为GLM-5 是 MLA+DSA, sglang 侧 is_mla_backend=True:
- KV latent 复制 ×8（每 prefill rank → 全部 8 个 decode rank）；
- DSA indexer state 只由 prefill CP rank0 发送（
mooncake/conn.py:1216-1234），×8 份。
又因为我们的开了 MC_ENABLE_DEST_DEVICE_AFFINITY=1 同名匹配 → 这些流量在 decode 侧全部从同名的 mlx5_000/001 进入；
def _get_dsa_cache_transfer_skip_flags(
self, info: Optional[KVArgsRegisterInfo]
) -> Tuple[bool, bool]:
skip_kv = False
skip_state = False
# Must be checked before the non-hybrid early return below, or every CP
# rank re-sends the same state and we transfer it cp_size times over.
# Prefill CP all-gathers before writing the pool, so every CP rank holds
# the full state regardless of whether the pool is hybrid. We assume no
# structure about the state rows, so we don't split them across CP ranks
# -- just let rank 0 send the whole thing (unless layer split already
# shards it per rank).
if (
self.attn_cp_size > 1
and self.attn_cp_rank != 0
and not get_parallel().enable_dsa_cache_layer_split
):
skip_state = True
if not self.is_hybrid_mla_backend:
return skip_kv, skip_state
if info is not None and self.attn_tp_size > info.dst_attn_tp_size:
sub_rank = (self.kv_args.engine_rank % self.attn_tp_size) % (
self.attn_tp_size // info.dst_attn_tp_size
)
if sub_rank != 0:
skip_kv = True
# Hybrid-MLA KV is replicated across these source ranks, but
# TP-sharded state needs every rank for the aggregation path.
return skip_kv, skip_state5 分析瓶颈（>100%）
要想进一步提升收益，在统计了各种参数组合（开关rr，调整上层 batch put 的线程数及并发数等等）提升都是有限。从代码层面进一步猜测 1：sglang 的传输本身就是全都是小 size 导致我们 pxn 的缺点（时延）进一步放大进而全串行。
| CHUNK | 1M PXN | 8M PXN | 512KB | direct |
|---|---|---|---|---|
| 64KB | 2.71 | 2.54 | 2.10 | 18.0 |
| 512KB | 10.8 | 10.6 | 11.51 | 21.1 |
| 1MB | 15.4 | 15.0 | 11.81 | 22.0 |
| 2MB | 16.7 | 21.7 | - | 22.1 |
| 4MB | 17.3 | 23.0 | - | 22.2 |
| 8MB | 17.7 | 23.7 | 12.17 | 21.8 |
1k1k
[2026-09-01 05:27:49 ATTN_CP1 TP1] Mooncake batch_transfer_sync: src_addrs_count=11, total_bytes=9792 (0.01 MiB), avg_length=0.87 KiB, min_length=64, max_length=8192, lengths=(64, 64, 64, 64, 512, 512, 64, 128, 64, 8192, 64)
[2026-09-01 05:28:23 ATTN_CP5 TP5] Mooncake batch_transfer_sync: src_addrs_count=78, total_bytes=184025088 (175.50 MiB), avg_length=2304.00 KiB, min_length=2359296, max_length=2359296, lengths=(2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296, 2359296)
8k1k
[2026-09-01 05:28:46 ATTN_CP6 TP6] Mooncake batch_transfer_sync: src_addrs_count=11, total_bytes=9792 (0.01 MiB), avg_length=0.87 KiB, min_length=64, max_length=8192, lengths=(64, 64, 64, 64, 512, 512, 64, 128, 64, 8192, 64)
[2026-09-01 05:28:46 ATTN_CP0 TP0] Mooncake batch_transfer_sync: src_addrs_count=78, total_bytes=336720384 (321.12 MiB), avg_length=4215.75 KiB, min_length=4316928, max_length=4316928, lengths=(4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928, 4316928)
64k1k
[2026-09-01 05:31:50 ATTN_CP0 TP0] Mooncake batch_transfer_sync: src_addrs_count=11, total_bytes=9792 (0.01 MiB), avg_length=0.87 KiB, min_length=64, max_length=8192, lengths=(64, 64, 64, 64, 512, 512, 64, 128, 64, 8192, 64)
[2026-09-01 05:31:50 ATTN_CP0 TP0] Mooncake batch_transfer_sync: src_addrs_count=78, total_bytes=674758656 (643.50 MiB), avg_length=8448.00 KiB, min_length=8650752, max_length=8650752, lengths=(8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752, 8650752)