fix(readme): the tier table disagreed with the router it documents - #223
Merged
Conversation
Three of twelve cells named a model the router does not route to: ECO SIMPLE said free/mistral-large-3-675b — config: free/gpt-oss-120b ECO REASONING said deepseek-reasoner — config: xai/grok-4-1-fast-reasoning AUTO REASONING same Nothing caught it because a README is prose to everything except a reader. src/router/brand-numbers.test.ts now reads the table and compares every cell to DEFAULT_ROUTING_CONFIG, so the doc fails the build rather than the reader. Separately, three places told users to pin `free/mistral-large-3-675b` as one of the eight free models. It is not one of the eight — it is hidden from /v1/models over NVIDIA's prompt-retention terms, so a reader following that instruction is pinning a model they cannot find in the catalog. Swapped for free/mistral-nemotron, which is genuinely in the published set, and dropped Mistral Large 3 from the "incl." list for the same reason. The corrected table now names two withheld primaries, so they carry a footnote saying so. That is the honest shape: the router does call them, and a reader should not go looking for them on the pricing page. It is also why the published savings figure is priced on visible models only.
📝 WalkthroughWalkthroughThe README updates free-model examples, routing profile documentation, and tier model mappings. A new test parses the README tier table and verifies its primary model names against ChangesRouting documentation alignment
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
src/router/brand-numbers.test.ts (1)
129-130: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winRemove the unchecked
Record<string, { primary: string }>cast.
premiumTiersis a realRoutingConfigkey, and each tier object has aprimaryproperty, so this assertion is not needed. It still bypasses stricter indexed-access checking and keeps the README assertor tied to a runtimeundefinederror instead of a type error if a tier orprimaryfield is renamed or removed. Use asatisfieshelper, a narrowed profile type, or the appropriate type guard instead ofas Record<...>.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/router/brand-numbers.test.ts` around lines 129 - 130, Remove the unchecked Record<string, { primary: string }> assertion from the DEFAULT_ROUTING_CONFIG lookup in the test. Use the existing RoutingConfig/premiumTiers typing, or introduce a satisfies-based helper, narrowed profile type, or type guard so tier and primary access remain statically validated without weakening indexed-access checks.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@README.md`:
- Around line 191-198: Fix the Markdown table row for `/model auto` by keeping
its entire Savings cell, including the savings marker and explanatory text, on a
single physical line with all four columns intact. Preserve the existing savings
content and ensure the `Best For` value `General use` remains part of the same
table row, eliminating the orphaned continuation line.
In `@src/router/brand-numbers.test.ts`:
- Around line 114-117: Update the table boundary in the brand-numbers test
around readme and table to use the actual README footer marker, matching the Ҡ
Withheld from /v1/models …” line or another explicit table-footer delimiter.
Ensure the marker is found before slicing so the complete tier table is
retained.
---
Nitpick comments:
In `@src/router/brand-numbers.test.ts`:
- Around line 129-130: Remove the unchecked Record<string, { primary: string }>
assertion from the DEFAULT_ROUTING_CONFIG lookup in the test. Use the existing
RoutingConfig/premiumTiers typing, or introduce a satisfies-based helper,
narrowed profile type, or type guard so tier and primary access remain
statically validated without weakening indexed-access checks.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: dcaeb568-d2b7-4b6b-8041-1f97354937ff
📒 Files selected for processing (2)
README.mdsrc/router/brand-numbers.test.ts
Comment on lines
+191
to
+198
| | Profile | Strategy | Savings | Best For | | ||
| | ------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- | | ||
| | `/model free` | Free NVIDIA models | **100%** | $0 balance, learning | | ||
| | `/model auto` | Balanced (default) | † Withheld from `/v1/models` — the router still calls it by direct ID, but you will not find it on the public pricing page. See [savings-mix.json](https://github.com/BlockRunAI/blockrun/blob/main/src/brand/savings-mix.json), which prices the published savings claim on visible models only. | | ||
|
|
||
| **<!-- br:savings.autoVsBaselinePct -->87<!-- /br:savings.autoVsBaselinePct -->%** | General use | | ||
| | `/model eco` | Cheapest possible | **<!-- br:savings.ecoVsBaselinePct -->98<!-- /br:savings.ecoVsBaselinePct -->%** | Maximum savings | | ||
| | `/model premium` | Best quality | 0% | Mission-critical | |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Broken markdown table — /model auto row spans multiple lines.
The Savings cell for /model auto contains an embedded line break (rows 194-196), which ends the GFM table early: the auto row loses its "Best For" cell, and the orphaned **87%** | General use | text on line 196 renders as loose paragraph text instead of a table row. Static analysis flags this exact spot (MD056, expected 4 columns, got 3 at line 194).
🩹 Proposed fix — collapse the row onto a single line
-| `/model auto` | Balanced (default) | † Withheld from `/v1/models` — the router still calls it by direct ID, but you will not find it on the public pricing page. See [savings-mix.json](https://github.com/BlockRunAI/blockrun/blob/main/src/brand/savings-mix.json), which prices the published savings claim on visible models only. |
-
-**<!-- br:savings.autoVsBaselinePct -->87<!-- /br:savings.autoVsBaselinePct -->%** | General use |
+| `/model auto` | Balanced (default) | **<!-- br:savings.autoVsBaselinePct -->87<!-- /br:savings.autoVsBaselinePct -->%** † Withheld from `/v1/models` — see [savings-mix.json](https://github.com/BlockRunAI/blockrun/blob/main/src/brand/savings-mix.json) | General use |📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| | Profile | Strategy | Savings | Best For | | |
| | ------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- | | |
| | `/model free` | Free NVIDIA models | **100%** | $0 balance, learning | | |
| | `/model auto` | Balanced (default) | † Withheld from `/v1/models` — the router still calls it by direct ID, but you will not find it on the public pricing page. See [savings-mix.json](https://github.com/BlockRunAI/blockrun/blob/main/src/brand/savings-mix.json), which prices the published savings claim on visible models only. | | |
| **<!-- br:savings.autoVsBaselinePct -->87<!-- /br:savings.autoVsBaselinePct -->%** | General use | | |
| | `/model eco` | Cheapest possible | **<!-- br:savings.ecoVsBaselinePct -->98<!-- /br:savings.ecoVsBaselinePct -->%** | Maximum savings | | |
| | `/model premium` | Best quality | 0% | Mission-critical | | |
| | Profile | Strategy | Savings | Best For | | |
| | ------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- | | |
| | `/model free` | Free NVIDIA models | **100%** | $0 balance, learning | | |
| | `/model auto` | Balanced (default) | **<!-- br:savings.autoVsBaselinePct -->87<!-- /br:savings.autoVsBaselinePct -->%** † Withheld from `/v1/models` — see [savings-mix.json](https://github.com/BlockRunAI/blockrun/blob/main/src/brand/savings-mix.json) | General use | | |
| | `/model eco` | Cheapest possible | **<!-- br:savings.ecoVsBaselinePct -->98<!-- /br:savings.ecoVsBaselinePct -->%** | Maximum savings | | |
| | `/model premium` | Best quality | 0% | Mission-critical | |
🧰 Tools
🪛 markdownlint-cli2 (0.23.0)
[warning] 194-194: Table column count
Expected: 4; Actual: 3; Too few cells, row will be missing data
(MD056, table-column-count)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@README.md` around lines 191 - 198, Fix the Markdown table row for `/model
auto` by keeping its entire Savings cell, including the savings marker and
explanatory text, on a single physical line with all four columns intact.
Preserve the existing savings content and ensure the `Best For` value `General
use` remains part of the same table row, eliminating the orphaned continuation
line.
Source: Linters/SAST tools
Comment on lines
+114
to
+117
| const readme = readFileSync("README.md", "utf8"); | ||
| // End at the footnote line, not the first "†" — that character also appears | ||
| // inside the table, marking primaries withheld from /v1/models. | ||
| const table = readme.slice(readme.indexOf("| Tier "), readme.indexOf("\n† Withheld")); |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Confirm the "† Withheld" footnote line exists as a standalone line in README.md
rg -n "^† Withheld" README.mdRepository: BlockRunAI/ClawRouter
Length of output: 159
🏁 Script executed:
#!/bin/bash
set -u
echo "README existence/size:"
[ -f README.md ] && wc -l README.md || echo "README.md not found"
echo
echo "Occurrences of Withheld or footnote dagger symbols:"
rg -n "Withheld|†|With*" README.md 2>/dev/null || true
echo
echo "Relevant README slices around likely table/footnote content:"
python3 - <<'PY'
from pathlib import Path
p=Path('README.md')
if not p.exists():
print("missing README.md")
raise SystemExit
s=p.read_text(encoding='utf-8', errors='replace')
terms=["Withheld","Withheld","†","| Tier "]
for term in terms:
idx=s.find(term)
print(f"\n--- first occurrence of {term!r} at {idx} ---")
if idx != -1:
line=s.count('\n',0,idx)+1
print(f"line {line}")
start=max(0,s.rfind('\n',0,idx-2000))
end=s.find('\n',idx+80)
if end==-1: end=len(s)
for ln,i in enumerate(range(start,end), start=idx):
print(f"{s[i:i+140]}")
if i>idx+200: break
PY
echo
echo "Search exact substring used by test:"
python3 - <<'PY'
import subprocess
res=subprocess.run(["rg","-n","\\n† Withheld","README.md"], text=True, capture_output=True)
print(res.stdout.strip() or "(no match)")
PYRepository: BlockRunAI/ClawRouter
Length of output: 50379
🏁 Script executed:
#!/bin/bash
set -u
python3 - <<'PY'
from pathlib import Path
import subprocess
p = Path('src/router/brand-numbers.test.ts')
if p.exists():
print("test_location_marker", p.resolve())
print(p.read_text(encoding='utf-8', errors='replace').splitlines()[100:125])
# Read-only behavioral probe for the exact JS slice behavior on current README and a set of representative footers.
readme = Path('README.md').read_text(encoding='utf-8', errors='replace') if Path('README.md').exists() else "<no README>"
def current_slice(text):
start = text.find("| Tier ")
end = text.find("\n† Withheld")
if start == -1 or end == -1:
return {"start_found": "found", "end_found": end == -1 and "not found" or "found", "start": start, "end": end, "length": -1 if end == -1 else end-start, "preview": text[start:end+30].replace("\n", "\\n")[:120]}
return {"start_found": "found", "end_found": "found", "start": start, "end": end, "length": end-start, "preview": text[start:end+30].replace("\n", "\\n")[:120]}
samples = {
"current_readme": readme,
"standalone footnote with single newline": ("| Tier Table\n† Withheld\nOther", True),
"standalone footnote with heading style": ("| Tier Table\n# † Withheld\nOther", True),
"no footnote": ("| Tier Table\n", True),
"same text in table row": ("| Tier Table\n| † Withheld | primary |\n", True),
}
print("probe_current_start", readme.find("| Tier "))
print("probe_current_end", readme.find("\n† Withheld"))
print()
for k, (text, _) in samples.items():
print(k, current_slice(text))
PYRepository: BlockRunAI/ClawRouter
Length of output: 1474
🏁 Script executed:
#!/bin/bash
set -u
python3 - <<'PY'
from pathlib import Path
readme = Path('README.md').read_text(encoding='utf-8', errors='replace')
table_marker = '| Tier '
footnote_marker = '\n† Withheld'
start = readme.find(table_marker)
end = readme.find(footnote_marker)
print(f"table_marker_found={start >= 0}")
print(f"footnote_marker_found={end >= 0}")
print(f"probe_start={start}")
print(f"probe_end={end}")
print(f"probe_slice_length={'n/a' if end < 0 or start < 0 else end - start}")
print()
# Inspect nearby source text without rendering the whole README.
if start >= 0:
for n, line in enumerate(readme.splitlines()[max(0, readme[:start].count('\n')-2):readme[:start].count('\n')+20], start=readme[:start].count('\n')+1):
if 'Tier' in line or 'Withheld' in line or line.startswith('\n'):
print(f"{n}: {line}")
PY
echo
echo "Nearby lines by line number:"
readme_line_start=$(python3 - <<'PY'
from pathlib import Path
readme = Path('README.md').read_text(encoding='utf-8', errors='replace')
print(readme[:readme.find("| Tier ")].count('\n') + 1)
PY
)
start=$((readme_line_start))
end=$((start + 120))
sed -n "${start},${end}p" README.md
echo
echo "Find exact footnote text (case-insensitive and variant-aware):"
rg -n "Withheld|†" README.md | head -80Repository: BlockRunAI/ClawRouter
Length of output: 9135
🏁 Script executed:
#!/bin/bash
set -u
python3 - <<'PY'
from pathlib import Path
from ast import literal_eval
test = Path('src/router/brand-numbers.test.ts').read_text(encoding='utf-8', errors='replace')
# Extract only the readline variable and table slice.
for i, line in enumerate(test.splitlines(), 1):
if "const readme = readFileSync" in line:
print(f"readline_line={i}: {line.strip()}")
if "const table = readme.slice" in line:
print(f"table_line={i}: {line.strip()}")
# Show the current end-index value that the exact source expression produces on current README.
for line in test.splitlines():
if "readme.indexOf" in line and '"\\n† Withheld"' in line:
print("exact_expression_result=", literal_eval(line.strip()).find("\\n† Withheld"))
print()
print("end_index=-1 means slice uses Table → end of README because JS negative end is treated as 0, slice is 0:end")
from pathlib import Path
readme = Path('README.md').read_text(encoding='utf-8', errors='replace')
table_start = readme.find("| Tier ")
print("table_start=", table_start)
print("slice(table_start, -1) produces", table_start, ":", len(readme) - 1)
PYRepository: BlockRunAI/ClawRouter
Length of output: 992
Bound the tier table to an existing README marker.
readme.indexOf("\n† Withheld") is -1 because README.md has no line starting with that exact footnote text, so slice(readme.indexOf("| Tier "), -1) drops the final character instead of stopping at the intended table/footer boundary. Use the actual README marker that exists, such as ^\n† Withheld from /v1/models ..., or an explicit table footer.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/router/brand-numbers.test.ts` around lines 114 - 117, Update the table
boundary in the brand-numbers test around readme and table to use the actual
README footer marker, matching the “† Withheld from /v1/models …” line or
another explicit table-footer delimiter. Ensure the marker is found before
slicing so the complete tier table is retained.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three of twelve cells were wrong
config.tsroutes tofree/mistral-large-3-675bfree/gpt-oss-120bdeepseek-reasonerxai/grok-4-1-fast-reasoningdeepseek-reasonerxai/grok-4-1-fast-reasoningNothing caught it, because a README is prose to everything except a reader.
src/router/brand-numbers.test.tsnow reads the table and compares every cell againstDEFAULT_ROUTING_CONFIG, so the doc fails the build instead of the reader. Verified it catches the original drift: restoringfree/mistral-large-3-675bfails withECO SIMPLE names the configured primary.The quick start pointed at a model users cannot find
Three places told readers to pin
free/mistral-large-3-675bas one of the eight free models:It is not one of the eight. It is
hidden: truein the gateway catalog — withheld from/v1/modelsover NVIDIA's prompt-retention terms — so someone following that instruction pins a model that does not appear in the catalog they were just told to browse.The eight actually published are all
nvidia/*:deepseek-v4-flash,nemotron-3-nano-omni-30b-a3b-reasoning,qwen3-next-80b-a3b-instruct,mistral-nemotron,step-3.7-flash,seed-oss-36b,nemotron-nano-9b-v2,nemotron-nano-12b-v2-vl.Swapped the example for
free/mistral-nemotron, and dropped "675B Mistral Large 3" from theincl.list for the same reason.Withheld primaries now carry a footnote
The corrected table names two models that
/v1/modelsdoes not list, so it says so:That is the honest shape. The router does call them; a reader should not go hunting for them on the pricing page. It is also exactly why the published savings figure is priced on visible models only.
Not a routing bug
free/gpt-oss-120bisavailable: true, hidden: true— callable by direct ID, deliberately absent from the public catalog. The router works; only the documentation was wrong.Verification
tscclean · prettier clean · 722 tests pass ·sync-brand-numbers --checkcleanSummary by CodeRabbit
Documentation
Tests