fix(llm): read the first text block of a bedrock Converse response

[zhiyanliu]

Summary

Both bedrock call sites read only the first content block of a Converse response. Converse returns output.message.content as a list and does not promise a text block is first — reasoning-capable models emit reasoningContent ahead of the answer, and toolUse or future block types can precede it too. For those models the hard-coded index returned the .get("text", …) default on every call, so _parse_llm_json saw an empty object, _response_is_hollow reported a hollow result, finish_reason was rewritten to length, and the adaptive retry bisected the chunk.

Splitting cannot converge, because the position assumption fails identically at every chunk size. stopReason on those responses is end_turn — the model answered correctly and was billed for it.

Fixes #2287.

What changed

Selection now keys on the block's shape rather than its position, via one helper used by _call_bedrock (primary extraction) and the backend == "bedrock" branch of _call_llm (secondary dispatch). A response whose first block is already text — every non-reasoning model today — is unaffected.

The helper is deliberately model-agnostic: it asks whether a block carries non-empty text, never which model produced it, so it also covers block types Converse may add later.

Verification

Probing Converse directly with a reasoning model shows the shape the old code could not read:

stopReason: end_turn
num content blocks: 2
  block[0] keys=['reasoningContent']
  block[1] keys=['text']
content[0].get("text","{}") -> '{}'

Re-running extraction on a 48-document corpus, changing only the block selection:

	before	after
hollow response lines	on essentially every call	none
bisection to recursion depth 3 (max 3)	repeatedly	none
files reported as producing no nodes	17 of 48	none
output tokens	217,538	53,274
graph	2,080 nodes / 4,516 edges	2,126 nodes / 4,580 edges

Document-type nodes rose from 43 to 66 and concept nodes from 98 to 130, so the extra coverage is real rather than a re-labelling. The output-token drop is the wasted bisection disappearing.

Tests

Added coverage for the block-selection contract: a single text block behaves exactly as before; a leading reasoningContent, toolUse, unknown future block type, or whitespace-only text block is skipped in favour of the real one; an absent text block, empty list, non-list content, and missing output/message all fall back to the caller's default; non-dict blocks and non-string text values are tolerated without raising. One end-to-end test drives _call_bedrock with a reasoning-model response and asserts the result is neither empty nor reclassified as truncation — it fails if the call sites are reverted to indexing position 0.

pytest tests/ shows no new failures against v8 (the pre-existing test_security / test_ollama_retry_cap / test_labeling failures in my environment are unrelated — local DNS and network sandboxing — and are identical with and without this change). ruff check is clean on both touched files.

Happy to reshape the helper or move it next to the other bedrock helpers if you'd prefer a different placement.

[graphify-labs]

Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).

Graphify reviewed this change.

Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).

---

Graphify review — findings

This PR adds a helper _bedrock_response_text to graphify/llm.py that scans the Bedrock Converse response's content blocks and returns the first block containing non-empty text, rather than assuming the text is at index 0. It replaces two existing hard-indexed content extractions (in _call_bedrock and _call_llm) with calls to this helper. The tests/test_image_vision.py file gains a new section of unit and end-to-end tests covering the helper's block-selection, fallback, and malformed-input behavior.

No blocking issues surfaced. 1 lower-confidence candidate did not survive cross-model review.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 600 functions depend on the 195 node(s) this change touches.

Health — this change adds coupling hotspots:

• worse: _call_llm() — 7 callers, 15 callees
• worse: _call_bedrock() — 3 callers, 6 callees

Verification — 600 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

• verification_scope: 410 function(s) in the blast radius were not formally verified this run

· 2 more finding(s) on lines outside this diff (see the check run).

[zhiyanliu]

Noted on the coupling advisory — flagging that I read it and am leaving the shape as-is, so a reviewer doesn't have to wonder.

The +1 callee on _call_bedrock and _call_llm is the new helper itself. Before this change the same expression was inlined verbatim at both sites, differing only in the default ("{}" vs ""), so extracting it trades one duplicated expression for one shared function. The metric reads that as added coupling, but the alternative — fixing the block selection twice in place — leaves the duplication and the next Converse-shape change has to be made in two places again.

Inlining the loop at both sites would keep the callee counts flat and I'd rather not: that reintroduces exactly the duplication that let the two sites drift in the first place (one already defaulted differently). Happy to inline it if you'd prefer to hold the counts flat, or to move the helper next to the other bedrock helpers if placement is the concern.