feat: carry an audio prompt's transcript text#30
Merged
Conversation
An audio prompt was a bare `{ audio: vprompt_… }` ref — the words the
clip speaks lived only where it was authored and were dropped from the
document. So a viewer can play a clip but cannot show what it says.
Add an optional `transcript` to the audio prompt object: the words the
clip was synthesized from, carried for display (what a caller hears) and
for traces. Advisory only — playback still uses `audio`, never this —
and optional, so older documents and non-generated refs stay valid and
older readers ignore it.
- schema: `Prompt` audio variant gains optional `transcript` (regenerated
the TS model + bundled schema; the crate-local copy syncs via build.rs).
- helpers: new `Prompt::transcript()` / `promptTranscript()` return the
spoken words for either prompt kind. `as_text()` / `promptText()` are
unchanged — they still mean "text to synthesize" (audio → None), so
playback branching and the length cap are untouched. Naming `transcript`
(not `text`) avoids a footgun: `promptText()` returns None for an audio
prompt, so a `text` field sitting beside it would read as contradictory.
- conformance: a new valid case exercises the field present and absent.
Lets the platform emit the transcript alongside a generated clip and the
voice daemon display it next to playback.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Hj93UCb3RKjF6mBwzhsp9f
wavekat-eason
force-pushed
the
feat/prompt-transcript
branch
from
July 20, 2026 08:26
6b8d121 to
62e8ad8
Compare
wavekat-eason
marked this pull request as ready for review
July 20, 2026 08:32
Merged
wavekat-eason
added a commit
that referenced
this pull request
Jul 20, 2026
#30 added an optional `transcript` to the audio-prompt model (regenerated the TS model + bundled schema, added `promptTranscript()`), but the hand-written safe-subset parser in `parse.ts` was never taught the field. So `decodePrompt` both warned "unknown field \"transcript\" is ignored by the engine" and dropped it — `promptTranscript()` returned null for any audio prompt decoded via `parseFlow`. Teach `decodePrompt` the field: add it to the known-field list (no more spurious warning) and carry it onto the returned prompt (validated as a string, so a non-string transcript is an `expected_string` error). The Rust twin already carries it — serde ignores unknowns and `Audio` has the column — so this only closes the TS-side gap. Surfaced by the platform baking transcripts into published flows: every audio prompt logged a warning on re-parse. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Today an audio prompt in a flow document is a bare reference —
prompt: { audio: vprompt_… }. The words the clip actually speaks are dropped: they exist where the clip was authored, but nowhere in the document. So a read-only viewer can play a prompt clip yet cannot show what it says — there's no transcript to display alongside playback.The text already exists upstream (it's what the clip was synthesized from). This change gives the document a place to carry it.
What
Add an optional
transcriptto thePromptaudio variant — the words the clip was synthesized from, carried for display (what a caller hears) and for traces.schema/flow.v1.schema.json):Promptaudio object gains an optionaltranscript. Regenerated the TS model + bundled schema viapnpm gen; the crate-local copy syncs throughbuild.rs, andtypifyemitsPrompt::Audio { audio, transcript: Option<String> }with#[serde(default, skip_serializing_if)]so it round-trips and stays absent when unset.model_ext.rs,model.ts): newPrompt::transcript()/promptTranscript()return the spoken words for either prompt kind (a text prompt is its own transcript; an audio prompt returns itstranscript, orNonewhen omitted).as_text()/promptText()are unchanged — they still mean "text to synthesize" and returnNonefor an audio prompt. Playback branching and the 2000-char length cap are deliberately untouched; an audio transcript is display text, not something to synthesize or bound.Why
transcript, nottextpromptText()deliberately returnsnullfor an audio prompt (it means "text to synthesize"). A field also namedtextwould sit right beside a helper that returnsnullfor it —promptText(p) === nullwhilep.textis a non-empty string.transcriptremoves that contradiction, matches the accessor name, and reads as the user-facing concept ("the transcript of what the clip says"). It intentionally does not mirror the platform'ssource_textcolumn name, which describes provenance rather than the viewer-facing meaning.audio-prompt-transcript) exercisestranscriptpresent on one node and absent on another — additive, always structurally valid.Compatibility
Fully additive and optional:
transcript) stay valid.Downstream (separate PRs, not in this one)
This is the shared-contract half. To reach the screen it needs:
transcriptalongside a generated clip when it produces/freezes the flow document (source text already stored), and pick up the new@wavekat/flow-schema.wavekat-flow, readprompt.transcript()for each audio prompt, thread it into the flow-detail asset view, and render it next to the play control.Testing
cargo test -p wavekat-flow— 35 unit (incl. newas_text/transcript/YAML round-trip inmodel_ext) + 3 conformance, green.pnpm --filter @wavekat/flow-schema test— 86 pass (newprompt.test.ts; conformance corpus picks up the new fixture);typecheckclean.cargo fmt --all --check+cargo clippyclean.pnpm genproduces no drift.🤖 Generated with Claude Code