Skip to content

Port SGLang integration to v0.5.15.post1 APIs#8

Draft
rchalamala wants to merge 21 commits into
mainfrom
cursor/sglang-v0515-api-port-609f
Draft

Port SGLang integration to v0.5.15.post1 APIs#8
rchalamala wants to merge 21 commits into
mainfrom
cursor/sglang-v0515-api-port-609f

Conversation

@rchalamala

@rchalamala rchalamala commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • port Foundry hooks to SGLang v0.5.15.post1 split runner and full CUDA-graph backend APIs
  • persist complete ShapeKey identity in a versioned latest-only graph index
  • validate archive schema, exact package version, graph configuration, and per-rank watermarks before LOAD
  • preserve deterministic FlashInfer and DeepEP/FA3 SAVE-to-LOAD allocation lifecycles
  • harden VMM CUDA-IPC across processes with SCM_RIGHTS fd transport, relocated peer mappings, refcounts, and chunk generations
  • add and validate exact dense TP=2 × DP=2 four-GPU support

Validated scope

  • single-GPU Qwen3-1.7B with FlashInfer
  • regular DP=2 Qwen3-1.7B with FlashInfer
  • dense TP=2 × DP=2 Qwen3-8B with FlashInfer/NCCL IPC, custom all-reduce disabled
  • EP=2 and EP=4 Qwen3-30B-A3B-FP8 with DP-attention, DeepEP low-latency, DeepGEMM, and FA3

Other conventional TP topologies, PP, prefill graphs, and unvalidated advanced modes remain rejected.

Testing

  • 74 passed, 5 skipped focused SGLang tests; skips are the absent optional local SGLang package
  • all pre-commit, shell syntax, compile, and diff checks pass
  • installed sglang==0.5.15.post1 API contract passes on Modal
  • baseline → SAVE → fresh-process LOAD → query passes with text/token parity and no recapture for single GPU, DP=2, EP=2, EP=4, and TP=2 × DP=2
  • TP=2 × DP=2: explicitly routed DP0/DP1 queries pass; four rank archives; 36 graphs/rank; identical final offsets; two NCCL TP groups; no IPC/CUDA/preallocation diagnostics
  • EP=4 was revalidated on the shipping hardened IPC hook: four rank archives; 20 graphs/rank; DeepEP/NVSHMEM/adapter evidence; 416 restored event nodes; no native diagnostics

The broader existing suite has unrelated baseline/environment failures documented in the validation report.

Open in Web Open in Cursor 

cursoragent and others added 21 commits July 23, 2026 06:15
Co-authored-by: Rahul Chalamala <rchalamala@users.noreply.github.com>
Co-authored-by: Rahul Chalamala <rchalamala@users.noreply.github.com>
Co-authored-by: Rahul Chalamala <rchalamala@users.noreply.github.com>
Co-authored-by: Rahul Chalamala <rchalamala@users.noreply.github.com>
Co-authored-by: Rahul Chalamala <rchalamala@users.noreply.github.com>
Co-authored-by: Rahul Chalamala <rchalamala@users.noreply.github.com>
Co-authored-by: Rahul Chalamala <rchalamala@users.noreply.github.com>
Co-authored-by: Rahul Chalamala <rchalamala@users.noreply.github.com>
Co-authored-by: Rahul Chalamala <rchalamala@users.noreply.github.com>
Co-authored-by: Rahul Chalamala <rchalamala@users.noreply.github.com>
Co-authored-by: Rahul Chalamala <rchalamala@users.noreply.github.com>
Co-authored-by: Rahul Chalamala <rchalamala@users.noreply.github.com>
Co-authored-by: Rahul Chalamala <rchalamala@users.noreply.github.com>
Co-authored-by: Rahul Chalamala <rchalamala@users.noreply.github.com>
Co-authored-by: Rahul Chalamala <rchalamala@users.noreply.github.com>
Co-authored-by: Rahul Chalamala <rchalamala@users.noreply.github.com>
Co-authored-by: Rahul Chalamala <rchalamala@users.noreply.github.com>
Co-authored-by: Rahul Chalamala <rchalamala@users.noreply.github.com>
Co-authored-by: Rahul Chalamala <rchalamala@users.noreply.github.com>
Co-authored-by: Rahul Chalamala <rchalamala@users.noreply.github.com>
Co-authored-by: Rahul Chalamala <rchalamala@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants