perf: reduce long session streaming render cost#450
Open
hsteude wants to merge 3 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR reduces UI rendering overhead during long, streaming sessions by minimizing allocations and avoiding rebuilding message/turn structures on every token update.
Changes:
- Cache
DisplayMessagewrappers in the session page so message objects aren’t recreated on each reactive update. - Apply streaming part deltas/updates in-place within the sync store to avoid rebuilding full arrays.
- Make turn/timeline and per-message derived UI data more reactive while aiming to reuse stable references.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| app-prefixable/src/pages/session.tsx | Adds DisplayMessage wrapper caching and cache eviction for non-visible messages. |
| app-prefixable/src/context/sync.tsx | Switches part update/delta handling to in-place store mutations via produce(). |
| app-prefixable/src/components/message-turn.tsx | Uses memos for derived assistant message content (text/tools/meta/subtasks) to react to streaming updates. |
| app-prefixable/src/components/message-timeline.tsx | Introduces structure comparison logic intended to reuse turn refs when only text changes. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+147
to
154
| const turnRefState = createMemo<TurnRefState>((prev) => { | ||
| const messages = props.messages | ||
| const structure = messageStructure(messages) | ||
| if (prev && sameTimelineStructure(prev.structure, structure)) { | ||
| return { structure, refs: prev.refs } | ||
| } | ||
| return { structure, refs: messagesToTurnRefs(messages.filter(hasStructuredContent)) } | ||
| }) |
Comment on lines
+277
to
+280
| get time() { | ||
| if (msg.info.role === "assistant") return { created: msg.info.time.created, completed: msg.info.time.completed }; | ||
| return { created: msg.info.time.created }; | ||
| }, |
Comment on lines
+143
to
150
| const turnRefState = createMemo<TurnRefState>((prev) => { | ||
| const messages = props.messages | ||
| const structure = messageStructure(messages) | ||
| if (prev && sameTimelineStructure(prev.structure, structure)) { | ||
| return { structure, refs: prev.refs } | ||
| } | ||
| return { structure, refs: messagesToTurnRefs(messages.filter(hasStructuredContent)) } | ||
| }) |
Comment on lines
+23
to
+28
| type MessageStructure = { | ||
| id: string | ||
| role: DisplayMessage["role"] | ||
| error: boolean | ||
| partCount: number | ||
| } |
Comment on lines
+86
to
+93
| function messageStructure(messages: DisplayMessage[]) { | ||
| return messages.map((msg) => ({ | ||
| id: msg.id, | ||
| role: msg.role, | ||
| error: !!msg.error, | ||
| partCount: msg.parts.length, | ||
| })) | ||
| } |
Comment on lines
+95
to
106
| function sameTimelineStructure(prev: MessageStructure[], next: MessageStructure[]) { | ||
| if (prev.length !== next.length) return false | ||
|
|
||
| for (let i = 0; i < next.length; i++) { | ||
| const a = prev[i] | ||
| const b = next[i] | ||
| if (a.id !== b.id || a.role !== b.role || a.error !== b.error) return false | ||
| if (a.partCount !== b.partCount) return false | ||
| } | ||
|
|
||
| return true | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Verification