Agent Effects

Agent Effects is a native, declarative motion-graphics renderer for AI coding agents. Projects are YAML data files with the .afx.yaml extension; the Rust binary interprets them without per-video compilation.

Quick start

cargo run -- init project.afx.yaml
cargo run -- validate project.afx.yaml
cargo run -- frame project.afx.yaml --time 2 --output frame.png
cargo run -- contact-sheet project.afx.yaml --output contact-sheet.png
cargo run -- render project.afx.yaml --output output.mp4
cargo run -- feedback project.afx.yaml
cargo run -- review project.afx.yaml

macOS editor

apps/macos contains the native SwiftUI/AppKit editor derived from Palmier Pro. It opens .afx.yaml directly and uses a persistent afx studio helper for timeline snapshots, atomic edits, diagnostics, playback frames and export. The Rust CLI remains the same executable on Linux and macOS. The app also exposes those operations through a loopback-only MCP server; it does not register Palmier account, generation, or cloud tools.

See apps/macos/README.md for the editor overview, apps/macos/AGENT_EFFECTS_BUILD.md for the current Mac build and test workflow, docs/studio-protocol.md for the frontend protocol, and docs/macos-mcp.md for local MCP setup.

The current vertical slice supports versioned projects, modular scene includes, story beats, semantic text/statistic/list/comparison/image/browser/photo/map compositions, deterministic frame evaluation, SVG ID inspection and rasterization, diagnostics, contact sheets, direct raw-frame FFmpeg encoding, narration muxing, word-marker timing, resumable chunk manifests, an editable native macOS timeline, and Gemini-backed video feedback.

Use cargo run -- docs question, cargo run -- schema, and cargo run -- examples search map to discover the language from an agent. Run cargo run -- preview project.afx.yaml before opening the review UI.

Frame, contact-sheet, preview and render commands accept --backend auto|cpu|gpu. auto selects a hardware Vulkan adapter when one is available and otherwise uses the CPU renderer. Forced gpu mode rejects software Vulkan adapters so benchmarks cannot accidentally report llvmpipe as GPU acceleration.

cargo run -- render project.afx.yaml --backend gpu --output output.mp4

GPU video renders convert directly to NV12 on the GPU before sending frames to FFmpeg, reducing readback and pipe bandwidth. Final renders use x264's medium preset by default; previews use ultrafast. Encoder and quality controls can be overridden:

cargo run -- render project.afx.yaml --backend gpu \
  --encoder x264 --encoder-preset ultrafast --crf 23 --output output.mp4

AMD VA-API hardware encoding is available with --encoder vaapi when mesa-va-drivers is installed. In VA-API mode, --crf is passed to the hardware encoder as its constant quantizer. auto currently selects x264 because raw-frame transfer to FFmpeg makes VA-API slightly slower on the tested Radeon 680M, although VA-API substantially lowers CPU use.

Interactive video renders display an in-place progress bar with completed frames, percentage, effective FPS, elapsed time and estimated time remaining.

On Linux, the rendering user needs read/write access to the DRM render node, normally through the render group.

Video feedback

critique reads the project file and contact-sheet shows stills. Neither can tell you whether the motion works. feedback renders the cut, sends it to Gemini along with the project's declared intent — beats, scene timings, presets and whether the renderer actually implements them, holds, theme — and returns a structured review.

export GEMINI_API_KEY=...
cargo run -- feedback project.afx.yaml
cargo run -- feedback project.afx.yaml --scene closing --json --output review.json

Findings come back anchored to a timestamp and the scene id that owns it, so a note about something at 12.4s names the scene to edit rather than a number to look up. The video is sampled at a rate derived from the shortest motion the project declares in the reviewed range, so a half-second entrance is not mistaken for a plain cross-fade; override it with --sample-fps. Rendered cuts are cached under .agent-effects/feedback/, keyed on the content of the project and every scene file it includes, so reviewing an unchanged project does not re-render it.

The key is read from --api-key, GEMINI_API_KEY, GOOGLE_API_KEY or ~/.config/agent-effects/.env — note that last path, which is the engine's own and not shared with any other tool. The model defaults to gemini-3-flash-preview and can be set with --model or AFX_GEMINI_MODEL. The upload is deleted after the review unless --keep-upload is given.

This is the only command that uses the network. Every other subcommand runs locally, and asset generation is deliberately kept outside the binary to keep it that way — see below.

Generating cutout assets

tools/afx-asset generates illustrations with a real alpha channel through Replicate, for projects that need artwork rather than type.

tools/afx-asset generate "laptop illustration" -o assets/laptop.png

It appends the transparency phrasing, checks the returned image actually has an alpha channel, retries when it comes back opaque, trims to the subject and writes a .asset.json sidecar recording prompt, model and seed. Transparency is stochastic — the same prompt returns a clean cutout one call and a flat rectangle the next — so the check, not the prompt, is what makes it usable.

It needs REPLICATE_API_KEY, in the environment or in ./.env. It is a script rather than a subcommand on purpose: the renderer stays local, and generating artwork is a studio step that happens before a project renders.

Reference studies

Engine capabilities are built from real reference videos rather than guesswork. One video is recreated in Agent Effects, and every shot the vocabulary cannot describe becomes a ranked gap in studies/GAPS.md. The process, including what never enters the repository, is docs/reference-studies.md.

tools/afx-shots reads a reference and prints its beat boundaries with hold durations, writing a still from each:

tools/afx-shots reference.webm --to 90 --stills

Reference videos, and every frame extracted from them, live in references/ and are gitignored. A study commits only text we wrote and assets we made.

Project guidance

Edit project.afx.yaml and files under scenes/. Prefer semantic compositions and named presets. After editing, validate first, render a frame or contact sheet, then run critique before a final render. Run feedback when the remaining question is about motion.

License

Agent Effects is licensed under the GNU General Public License version 3. The macOS editor under apps/macos is derived from Palmier Pro and retains its upstream copyright and license notices. See NOTICE.md and apps/macos/UPSTREAM.md.