Skip to content

feat: apply marketplace price multipliers#64

Merged
jmlago merged 2 commits into
mainfrom
codex/marketplace-price-multipliers
Jul 1, 2026
Merged

feat: apply marketplace price multipliers#64
jmlago merged 2 commits into
mainfrom
codex/marketplace-price-multipliers

Conversation

@MuncleUscles

@MuncleUscles MuncleUscles commented Jul 1, 2026

Copy link
Copy Markdown
Member

Summary

  • default bedrock.price_multiplier to 0.8 and openrouter.price_multiplier to 1.05
  • keep marketplace/source offers raw, then expose effective_price_* plus ranking_price_multiplier at discovery time for selection
  • let marketplace providers inherit the source multiplier, e.g. bedrock_market uses bedrock.price_multiplier
  • invalidate marketplace discovery on config reload so multiplier changes apply immediately
  • keep request cost accounting on the raw quote by dividing out the carried ranking multiplier

Engine dependency

Validation

  • PYTHONPYCACHEPREFIX=/private/tmp/unhardcoded-pycache .venv/bin/python -m pytest tests/test_sources.py::test_push_prices_only_pushes_cataloged_pairs tests/test_sources.py::test_push_prices_applies_effective_price_multiplier tests/test_sources.py::test_discover_hook_adds_effective_offer_prices_without_mutating_raw tests/test_providers.py::test_priced_providers_get_an_effective_multiplier_knob tests/test_shim.py::test_executed_cost_usd_ignores_the_ranking_multiplier tests/test_live_wiring.py::test_marketplace_offers_rank_with_effective_prices_when_present -q — 6 passed
  • git diff --check
  • PYTHONPYCACHEPREFIX=/private/tmp/unhardcoded-pycache .venv/bin/python -m py_compile providers.py serve.py shim.py tests/test_sources.py tests/test_live_wiring.py tests/test_shim.py tests/test_providers.py

Note on broader local tests

  • A broader local run over tests/test_sources.py tests/test_providers.py tests/test_shim.py tests/test_live_wiring.py reached 63 passed, 19 skipped with no failures but stalled until interrupted at ~25 minutes. The slow path appears to be unrelated app/lifespan/live-wiring setup, not the focused multiplier cases.

Summary by CodeRabbit

  • New Features

    • Marketplace offers now surface effective ranking prices when a price multiplier is configured, improving how offers are ordered without changing raw quoted prices.
    • Provider pricing reloads refresh marketplace discovery immediately, so updated pricing can appear faster.
  • Bug Fixes

    • Billing cost calculations now ignore ranking-only multipliers, helping ensure spend is based on the underlying quoted price.
    • Marketplace offer handling now preserves raw offer data while using adjusted prices for ranking.

@coderabbitai

coderabbitai Bot commented Jul 1, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Updates the core submodule pointer and introduces effective ranking-price handling for marketplace offers: the discover hook computes effective prices from a per-provider price_multiplier without mutating raw offers, billing divides the multiplier back out, and config reload invalidates marketplace discovery caches. Documentation and tests are updated accordingly.

Changes

Marketplace Effective Pricing

Layer / File(s) Summary
Price multiplier knob documentation
providers.py, sources/__init__.py, tests/test_providers.py
Clarifies help text and docstrings describing that price_multiplier is a ranking-only lever, keeping raw quotes intact and unaffected billing; reformats an existing test assertion.
Discover hook effective offer pricing
serve.py, tests/test_live_wiring.py, tests/test_sources.py
make_discover_hook tracks per-provider source names, derives multipliers from settings, and computes effective_price_in/out_usd_per_mtok plus ranking_price_multiplier on returned offers without mutating the raw offer; adds corresponding e2e and unit tests.
Billing reconciliation and discovery invalidation
shim.py, tests/test_shim.py
_executed_cost_usd validates and divides out the effective multiplier when computing billing cost; reload_config now invalidates marketplace provider discovery caches after settings reload; adds a billing test scenario.
Core submodule pointer
core
Updates the tracked core subproject commit reference.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Core
  participant ServeHook as make_discover_hook
  participant Settings
  participant Source
  participant Shim as _executed_cost_usd

  Core->>ServeHook: discovery_id request
  ServeHook->>Source: offers_sync(discovery_id)
  Source-->>ServeHook: raw offers (unmutated)
  ServeHook->>Settings: lookup provider/source price_multiplier
  Settings-->>ServeHook: multiplier value
  ServeHook->>ServeHook: compute effective_price_in/out via _effective_offer
  ServeHook-->>Core: offers with ranking_price_multiplier
  Core->>Shim: chosen offer for cost calc
  Shim->>Settings: fallback lookup if price_multiplier invalid
  Shim->>Shim: divide out multiplier from price_in/price_out
  Shim-->>Core: raw billing cost_usd
Loading

Possibly related PRs

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 45.45% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: applying marketplace price multipliers.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/marketplace-price-multipliers

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

… core to merged engine #26

Align the code with the stated design: the price multiplier is a RANKING-only
lever, and a routing preference is an operator decision set + persisted from the
Config tab — not hardcoded. Drop _DEFAULT_PRICE_MULTIPLIERS ({bedrock:0.8,
openrouter:1.05}); every provider defaults to 1.0 (no nudge). A REAL per-call
surcharge is not a multiplier — it belongs in the provider's reported/list price
so it ranks AND bills; openrouter's 5% is a credit top-up fee, not per-call, so
openrouter needs no multiplier. This also removes the billing under-report the
hardcoded 1.05 caused (test_executed_cost_usd_rules: 3.0/1.05).

Re-pin core e8263d4 -> 38a79c6 (engine #26 squash-merged to engine main).
Tests aligned to the 1.0 default (test_providers, test_sources).

Full suite 446 passed / 2 skipped against compose Postgres.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
serve.py (1)

219-235: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Extend the try/except to cover per-offer effective-price mapping.

hook()'s docstring states it "must be fast and never raise" (called from Lua inside rank), and fn(discovery_id) is guarded accordingly. But the new _effective_offer mapping on Line 233 runs outside that try/except. If any offer from a source isn't a well-formed dict (or a future source bug), dict(offer) in _effective_offer raises, and it now escapes hook() uncaught.

🛡️ Proposed fix: widen the try/except scope
         try:
             offers = fn(discovery_id)
+            offers = [_effective_offer(o, discovery_id, source_name) for o in offers] if offers else offers
         except Exception as exc:  # noqa: BLE001
             return {"ok": False, "error": f"{type(exc).__name__}: {exc}"}
         if not offers:
             # not-ok so the core does NOT cache emptiness for the discovery
             # TTL — a router that starts before the first market dump should
             # pick offers up on the next rank, not minutes later.
             return {"ok": False, "error": "no offers"}
-        offers = [_effective_offer(o, discovery_id, source_name) for o in offers]
         return {"ok": True, "fetched_at_ms": int(time.time() * 1000),
                 "offers": offers}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@serve.py` around lines 219 - 235, The uncaught failure path in hook() comes
from the per-offer _effective_offer mapping running outside the existing
exception guard. Widen the try/except in hook() so it covers both
fn(discovery_id) and the list comprehension that applies _effective_offer to
each offer, ensuring any bad offer shape or dict(offer) failure is converted
into the same {"ok": False, "error": ...} response. Keep the existing unknown
discovery_id and no offers handling unchanged, and preserve the fast-never-raise
behavior described for hook().
🧹 Nitpick comments (1)
tests/test_sources.py (1)

1185-1191: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Mutable class attribute flagged by Ruff (RUF012).

provider_ids = ["bedrock_market"] is a mutable list default in the class body. Test-only fixture, so the risk is negligible, but a ClassVar annotation or tuple would satisfy the linter.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/test_sources.py` around lines 1185 - 1191, The FakeBedrockSource
fixture uses a mutable class attribute for provider_ids, which triggers Ruff
RUF012. Update the class-level provider_ids on FakeBedrockSource to use an
immutable ClassVar/tuple (or otherwise annotate it as a ClassVar) so the test
fixture still works while satisfying the linter. Keep the change localized to
the FakeBedrockSource definition and preserve the existing provider_id check in
offers_sync.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@serve.py`:
- Around line 219-235: The uncaught failure path in hook() comes from the
per-offer _effective_offer mapping running outside the existing exception guard.
Widen the try/except in hook() so it covers both fn(discovery_id) and the list
comprehension that applies _effective_offer to each offer, ensuring any bad
offer shape or dict(offer) failure is converted into the same {"ok": False,
"error": ...} response. Keep the existing unknown discovery_id and no offers
handling unchanged, and preserve the fast-never-raise behavior described for
hook().

---

Nitpick comments:
In `@tests/test_sources.py`:
- Around line 1185-1191: The FakeBedrockSource fixture uses a mutable class
attribute for provider_ids, which triggers Ruff RUF012. Update the class-level
provider_ids on FakeBedrockSource to use an immutable ClassVar/tuple (or
otherwise annotate it as a ClassVar) so the test fixture still works while
satisfying the linter. Keep the change localized to the FakeBedrockSource
definition and preserve the existing provider_id check in offers_sync.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 54b6e1da-2c29-40b0-8811-61dc09c2478a

📥 Commits

Reviewing files that changed from the base of the PR and between f0688cf and 2f818c2.

📒 Files selected for processing (9)
  • core
  • providers.py
  • serve.py
  • shim.py
  • sources/__init__.py
  • tests/test_live_wiring.py
  • tests/test_providers.py
  • tests/test_shim.py
  • tests/test_sources.py

@jmlago jmlago merged commit c7e1cee into main Jul 1, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants