dsv4 sglang agentic: enable out-of-window SWA slot release and bump image to nightly-20260707

[cquil11]

Change

1. Update the SGLang image to a recent nightly

lmsysorg/sglang:v0.5.13-cu130 → lmsysorg/sglang:nightly-dev-cu13-20260707-b4155233
for the dsv4-fp4-b200-sglang-agentic-hicache and dsv4-fp4-b300-sglang-agentic-hicache
jobs. The newer build carries recent kernel and runtime updates

2. Enable two engine env vars in the b200/b300 agentic launch scripts

• SGLANG_ENABLE_UNIFIED_RADIX_TREE=1 — selects the unified radix tree, the cache
  implementation that supports per-component (full-attention / SWA) management for
  hybrid-attention models. It was previously exported only inside the hicache branch;
  it is now set unconditionally (the duplicate in-branch export is removed).

• SGLANG_OPT_UNIFIED_CACHE_FREE_OUT_OF_WINDOW_SLOTS=1 — proactively frees
  out-of-window SWA KV slots during chunked prefill. Without it, in-flight
  requests hold SWA KV for their entire context, which keeps the SWA pool under
  constant pressure and forces frequent evictions. Most importantly, under LRU
  the last-window KV of cached sessions gets flushed as well; since a prefix
  match is only valid when that trailing window is present, hits become
  bimodal — a session either matches almost fully or not at all — and the
  effective prefix-cache hit rate collapses on multi-turn agentic workloads.
  Enabling the release removes this pressure at the source.

Perf Comparison

Measured on an 8×B200 single-node reproduction of this agentic benchmark at concurrency 64 DEP =8, No Hicache (600 s trace replay):

Metric	Before	After
Actual prefix-cache hit	~45%	~74%
Total throughput (tok/s)	~40K	~106K
TTFT	~17 s	~6 s
Silent cache evictions	~3K/run	0

Note: reopening this directly rather than from a fork. Original author: @Oasis-Git

[claude]

🔴 This PR bumps the SGLang image tag for dsv4-fp4-b200-sglang-agentic-hicache (line 12683) and dsv4-fp4-b300-sglang-agentic-hicache (line 12705) but does not append a matching entry to perf-changelog.yaml. Per AGENTS.md's "Updating Docker images" section, a changelog entry is required — it's what triggers .github/workflows/run-sweep.yml (which filters on paths: - perf-changelog.yaml). Without it, neither the PR fail-fast sweep nor the post-merge push-to-main sweep will actually run for these two configs, so the ~40K→106K tok/s / TTFT 17s→6s improvements claimed in the PR description will not be reflected in tracked main-branch results.

Extended reasoning...

What the bug is

The PR modifies exactly 3 files: two launcher scripts (benchmarks/single_node/agentic/dsv4_fp4_b{200,300}_sglang.sh) and configs/nvidia-master.yaml. In configs/nvidia-master.yaml it bumps the image for two configs:

• dsv4-fp4-b200-sglang-agentic-hicache at line 12683: lmsysorg/sglang:v0.5.13-cu130 → lmsysorg/sglang:nightly-dev-cu13-20260707-b4155233
• dsv4-fp4-b300-sglang-agentic-hicache at line 12705: same bump

It also adds two new engine env vars (SGLANG_ENABLE_UNIFIED_RADIX_TREE=1 and SGLANG_OPT_UNIFIED_CACHE_FREE_OUT_OF_WINDOW_SLOTS=1) to both launcher scripts. However, perf-changelog.yaml is not touched at all.

Why this matters — the trigger contract

AGENTS.md (Updating Docker images) is explicit: on an image bump you must "append a perf-changelog.yaml entry (required - triggers benchmarks)". This is not just documentation hygiene — it is the actual trigger:

# .github/workflows/run-sweep.yml
on:
  push:
    branches: [main]
    paths:
      - perf-changelog.yaml
  pull_request:
    paths:
      - perf-changelog.yaml

Because the workflow's paths: filter only fires when perf-changelog.yaml is modified, this PR's diff will not cause run-sweep.yml to invoke — despite the full-sweep-fail-fast label being set. The label controls behavior inside the workflow once it runs; it does not cause the workflow to run in the first place.

Concrete step-by-step proof

1. Contributor pushes commits touching configs/nvidia-master.yaml and the two .sh files.
2. GitHub Actions receives the pull_request event.
3. Actions consults run-sweep.yml's on.pull_request.paths filter: perf-changelog.yaml — not in the changed file list.
4. Actions does not schedule the sweep workflow. No matrix job runs; no fail-fast gate exists.
5. PR merges (nothing gated it). push to main fires; on.push.paths filter again requires perf-changelog.yaml — again not present.
6. No post-merge sweep runs against these two configs on the new image. The tracked main-branch performance dashboards keep the old image's numbers.

Precedent — every recent bump followed this convention

Recent image-bump PRs (#2056, #2058, #2060, #2067, #2073, #2074, #2077) all touched both configs/*-master.yaml and appended a perf-changelog.yaml block. That's the workflow this repo relies on.

Fix

Append an entry to perf-changelog.yaml in the same style as the tail of that file — model+precision+SKU, PR number, one-line reason (e.g. "Bump sglang nightly to pick up unified radix-tree out-of-window SWA slot release"), and the two affected config keys (dsv4-fp4-b200-sglang-agentic-hicache and dsv4-fp4-b300-sglang-agentic-hicache). That single addition arms both the PR fail-fast sweep and the post-merge push-to-main sweep.