Skip to content

feat(render): Add RGB565 raster path and ESP32-P4 PPA backend#160

Merged
doodlewind merged 9 commits into
pocket-stack:mainfrom
HalfSweet:feat/esp32p4-ppa-rgb565
Jul 24, 2026
Merged

feat(render): Add RGB565 raster path and ESP32-P4 PPA backend#160
doodlewind merged 9 commits into
pocket-stack:mainfrom
HalfSweet:feat/esp32p4-ppa-rgb565

Conversation

@HalfSweet

@HalfSweet HalfSweet commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add opaque PSP PSM5650 texture support and a native RGB565 raster target
  • add an ESP32-P4 DrawList backend backed by PPA FILL, A8 BLEND, and SRM
  • preserve DrawList ordering with an RGB565 software fallback for unsupported operations
  • add a reusable ESP-IDF adapter, component smoke application, documentation, and CI
  • keep the complete rendering path in RGB565 without a full-frame RGB888 or ARGB8888 intermediate

Implementation

The new pocketjs-esp32p4-ppa backend interprets DrawLists directly into an RGB565 destination.

Hardware-compatible operations are routed to:

  • PPA FILL for opaque rectangles
  • PPA BLEND for translucent rectangles, antialiased glyphs, and white-alpha textures
  • PPA SRM for compatible opaque PSM5650 textures, including exact scaling, quarter-turn rotation, and mirroring

Unsupported gradients, triangles, texture formats, transforms, and sampling cases fall back to the native RGB565 software rasterizer in DrawList order.

The reusable ESP-IDF adapter:

  • registers separate FILL, BLEND, and SRM clients
  • uses blocking transactions so subsequent PPA and CPU operations observe completed pixels
  • expands RGB565 FILL colors through fill_argb_color
  • rejects overlapping SRM input/output buffers
  • accepts only scale factors represented exactly in the PPA's 1/16 increments
  • leaves display initialization, presentation buffers, and panel-specific behavior in the product BSP

Linear alpha-texture classification and edge sampling are shared with the core rasterizer. Colored transparent textures using linear filtering correctly fall back to software rendering.

Compatibility

  • ESP-IDF release/v6.0: supported and used as the CI baseline
  • ESP-IDF release/v6.1: supported and locally build-tested
  • versions older than v6.0 have not been tested
  • target: ESP32-P4

Hardware status

In the M5Stack Tab5 hardware integration build, the average frame rate reached 19.8 FPS during extended operation. The test was conducted at a CPU frequency of 360 MHz and a resolution of 1280 × 720.

@HalfSweet HalfSweet changed the title feat: Add esp32p4 PPA support feat(render): Add RGB565 raster path and ESP32-P4 PPA backend Jul 23, 2026
@HalfSweet
HalfSweet marked this pull request as ready for review July 23, 2026 09:53
Scrub ESP32-P4/PPA-specific wording from engine/core docs and comments —
the core raster targets are format contracts (RGBA8, LE-ARGB8888 words,
RGB565), not properties of one host. Export coverage_index alongside
linear_sample_coordinates and drop the backend's byte-for-byte duplicate,
so hardware glyph masks can never drift from the software fallback. Also
fix the pixel-count assert message left over from the byte-length days.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@doodlewind doodlewind left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the full diff, ran the complete local suite, and pushed one maintainer commit (7975a29) on top — details below. This is a model contribution: the hardware surface is isolated behind a narrow trait, the generic layers gain a format, not a platform, and the fallback story preserves DrawList ordering exactly.

What I verified

  • engine/core tests: 85 passed; backend tests: 10 passed; --features esp-idf cargo check clean.
  • contracts parity: bun tests/contract.ts green, and gen-rust.ts regenerates spec.rs byte-identical — the hand edit matches the generator.
  • Full bun run test chain green.
  • Goldens: 42 pass; the 7 failures (music/gallery/motions) reproduce identically on main with a fresh wasm rebuild in this environment, so they are pre-existing and not from this PR. The RenderTarget refactor keeps the RGBA path byte-exact.

Architecture notes (all good)

  • PpaOps keeps ESP-IDF headers out of the Rust crate entirely; the C component owns the clients and the DMA/cache contract. Board/BSP concerns stay out of the repo.
  • The alpha-texture classification cache is safe against handle reuse because core handles are generation-tagged, and T8 video planes are correctly excluded (in-place update_texture_t8 mutation).
  • Fractional-UV rejection (exact_texel_edge) and exact 1/16-scale admission are the right conservative gates — everything else falls back in order.
  • C side: rect/surface validation, SRM overlap rejection, fill_argb_color expansion, and log-once fallback are all careful.

One heads-up for downstream — @odex21

This PR changes render_scaled_argb output from [A,R,G,B] byte order to [B,G,R,A] (the little-endian layout of an ARGB8888 word, which is what the PPA actually consumes). If your #140 firmware compensated for the old order anywhere, it needs the inverse adjustment when it picks up this change. The new order is the correct one for feeding the PPA without a reorder pass.

Maintainer commit 7975a29

  • De-platformed engine/core docs/comments: core describes format contracts (RGBA8 / LE-ARGB8888 / RGB565), not one host's hardware; per-host rationale lives in engine/backends/esp32p4-ppa and hosts/esp32p4 where it belongs.
  • Exported coverage_index from pocketjs_core::raster (same rationale as your linear_sample_coordinates share) and removed the backend's duplicate, so glyph-mask sampling can never drift from the software fallback.
  • Fixed the stale "wrong byte length" assert message (it now checks pixel count).

Thanks for the thorough tests — the mock-PPA parity tests against render_scaled_rgb565 are exactly the right oracle. Merging once CI is green on the new head.

@doodlewind
doodlewind merged commit 76d5c48 into pocket-stack:main Jul 24, 2026
2 checks passed
@doodlewind

Copy link
Copy Markdown
Collaborator

Correction to my review's golden-suite note, for the record.

I wrote that the 7 golden failures (gallery ×5, music-main.60, motions-main.236) "reproduce identically on main" — that framing was wrong. The accurate diagnosis:

No action needed — the merge is clean. Lesson for future branches: goldens are only meaningful against the branch's own base or after rebasing onto current main.

@odex21

odex21 commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Real-device A/B/C benchmark on ESP32-P4 (AXS15260, 452×1280)

Thanks @HalfSweet for this work — the backend architecture (ordered PPA dispatch + software fallback + damage) is solid and runs tear-free on real hardware. We benchmarked it on our player UI and wanted to share the numbers, since the conclusion differs by UI shape.

Setup. Three builds on the same device and display chain, ESP-IDF v5.5.4, pocketjs @ d027222 (main + touch-events, i.e. post-#160). Panel always scans RGB888; RGB565 only feeds the PPA input, converted to RGB888 during the 2× scale + 90° rotate. Triple-buffered, vsync-owned. Sampling: serial vue.perf 1s-window medians, 30 s per scenario; "playing" gated on playback_ms advancing + audio_playing=true.

A (63ab281, ARGB) B (d027222, ARGB) C (d027222, RGB565)
playing (60s lyric scroll) fps 22.9 23.9 24.5
static fps 27.4 28.0 31.8
drag-ball fps 28.8 30.3 26.2
playing render 30.8 ms 29.3 ms 23.5 ms
static render 33.8 ms 33.1 ms 23.4 ms
drag-ball render 15.5 ms 13.7 ms 15.1 ms
present (PPA→panel) 0.5 ms 0.5 ms 6.0 ms

Invariants (drops / free_waits / ownership_violations) all 0 on every build — no tearing.

Observations.

  • B vs A (just picking up feat(render): Add RGB565 raster path and ESP32-P4 PPA backend #160's raster rework + the ARGB layout fix): render drops ~1.5 ms and present is identical, so the rgb_swap toggle is genuinely free (it is a hardware mux on the PPA read path). Pure win, no regression — we'll take it.
  • C vs B (RGB565): render drops 6–10 ms (halved PSRAM write bandwidth + PPA taking some fills/blends), which lifts static/playing fps. But present rises 0.5→6.0 ms (the extra SRM 565→888 conversion, paid every frame), and drag-ball fps drops 30.3→26.2 — the ball is all small rects that miss the PPA area thresholds, so it pays the 6 ms conversion with no raster win to offset it.
  • PPA dispatch rate: ppa_fill=2~5 ppa_blend=16~18 ppa_srm=0 sw_ops=36. The accelerator catches only a few large fills/blends; SRM never fires (our 92×92 cover is too small and doesn't meet the format/scale conditions), and the many small rects/glyphs fall back to software.
  • Banding: the slate-950 → indigo-950 root gradient shows clearly visible banding under RGB565 (5–6 bits/channel has no steps between two very close dark blues). RGB888 is clean. This is a bit-depth physical limit, not an implementation issue.

Takeaway for us. The +fps on static/playing is real, but it is offset by the per-frame 565→888 conversion and reversed on our drag-heavy page, SRM doesn't apply to our UI, and the gradient banding is a blocker (we plan heavier fluid gradients). The backend's payoff model fits "fullscreen image + large flat-color" UIs better than our "many small rects + small text + one small cover" player. So we won't adopt the RGB565 backend itself. That said, #160's raster rework (fill_opaque_span fast path) helps ARGB too — we measured ~1.5 ms lower render with zero cost, so we are taking just that part.

No action needed — just sharing real-device data in case it helps tune the PPA area thresholds or documents the RGB565 banding trade-off. Happy to provide the raw serial logs if useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants