runctl

A small, cross-platform (macOS + Windows) command-line tool for desktop control. It exposes a few primitive "computer-use" actions as subcommands so they can be driven from a script or an AI agent:

• list — list monitors with index, offset, size, scale, and primary flag
• shot — screenshot a monitor (or all monitors, the whole desktop, or one window) to PNG, optionally with a coordinate grid (--grid)
• click / move — move the cursor to (x, y) and click (monitor- or window-relative)
• activate — bring a window to the front (so a following click/type lands on it)
• scroll — scroll the wheel (optionally over a given point)
• drag — press at one point, drag to another, and release
• watch — watch a window or screen for changes and emit an event / run a command
• open — open a URL, file, or app with the system default handler
• type — type text at the current keyboard focus
• key — press a key or chord (e.g. enter, cmd+a) at the current focus
• check — report (and on macOS optionally request) permissions

Single binary, no runtime. Every command prints a one-line human summary, or machine-readable JSON with --json, and exits 0 on success / 1 on error.

Install

No environment variables to configure — every route below puts runctl on your PATH.

curl -fsSL https://raw.githubusercontent.com/runify-dev/runctl/main/install.sh | sh

(Or ./install.sh from a clone.) Pass options as env vars before sh, e.g. curl -fsSL .../install.sh | RUNCTL_FORCE_DOWNLOAD=1 sh.

install.sh picks a mode automatically:

• If cargo is available, it builds and installs from source into ~/.cargo/bin (already on PATH after a normal Rust install). Run from a clone it uses cargo install --path .; piped (curl -fsSL .../install.sh | sh) it uses cargo install --git.
• Otherwise (no cargo, macOS only) it downloads a prebuilt binary from the latest GitHub Release, verifies its SHA-256, and installs it (default /usr/local/bin, override with RUNCTL_INSTALL_DIR), clearing the Gatekeeper quarantine flag.

Useful knobs: RUNCTL_VERSION=v0.1.0 to pin a tag, RUNCTL_FORCE_DOWNLOAD=1 to skip cargo.

install.sh is a POSIX shell script (macOS/Linux). Other platforms:

• Windows — cargo install --git https://github.com/runify-dev/runctl, or download runctl-windows-x64.zip from the Releases page and put runctl.exe on your PATH. (There is no PowerShell installer yet.)
• Linux — no prebuilt binary is published; install Rust and run cargo install --git https://github.com/runify-dev/runctl.

Or do it by hand: cargo install --path ., or copy the built binary onto your PATH (cp target/release/runctl /usr/local/bin/).

On macOS, grant permissions before first use — see Permissions and run runctl check.

Build

Requires a Rust toolchain. Build on the platform you are targeting (you cannot cross-build macOS from Windows or vice-versa — the crates link against each OS's native frameworks).

cargo build --release
# binary at target/release/runctl  (runctl.exe on Windows)

If a dependency version fails to resolve, run cargo update — the versions in Cargo.toml are minimums.

Usage

runctl list                                # enumerate monitors

runctl shot --out shot.png                 # primary monitor
runctl shot --out shot.png --screen 1      # monitor index 1 (see `list`)
runctl shot --out shot.png --screen all    # one file per monitor: shot_0.png, shot_1.png, ...
runctl shot --out shot.png --screen full   # whole virtual desktop stitched into one image
runctl shot --out wx.png --window 微信      # one window (works even if it's covered by others)
runctl shot --out wx.png --window 微信 --grid            # overlay a coordinate grid + labels
runctl shot --out wx.png --window 微信 --grid edge --grid-step 50   # edge ticks only, 50px
runctl shot --out wx.png --window 微信 --grid --grid-step 200 --grid-coords  # label each intersection "x,y"

runctl move --x 500 --y 300                # global coordinates
runctl click --x 500 --y 300
runctl click --x 500 --y 300 --button right
runctl click --x 500 --y 300 --double
runctl click                               # click at the current cursor position (no move)

# coordinates measured off `shot --screen 1` (offset + DPI applied for you):
runctl click --screen 1 --x 480 --y 220

# click a spot you saw in `shot --window` — pass the screenshot's size so the
# pixel is mapped proportionally onto the window (correct at any DPI/scale):
runctl activate 微信                        # bring it to the front first
runctl click --window 微信 --x 440 --y 600 --shot-w 1280 --shot-h 720

# precise targeting by id (from `windows`) when several windows share a name:
runctl windows                              # list windows with their ids
runctl --json shot --window-id 51234 --out w.png        # note its width/height
runctl click --window-id 51234 --x 440 --y 600 --shot-w 1280 --shot-h 720
runctl activate --window-id 51234

runctl scroll --dir down --amount 3        # scroll the focused window down
runctl scroll --dir up --screen 1 --x 400 --y 300   # move over a pane, then scroll it
runctl scroll --dir up --window-id 51234 --x 100 --y 400 --shot-w 1280 --shot-h 720 --amount 8

runctl drag --from-x 100 --from-y 200 --to-x 400 --to-y 200          # global coords
runctl drag --screen 1 --from-x 100 --from-y 200 --to-x 400 --to-y 200 --button left

runctl open https://example.com
runctl open ./report.pdf
runctl type "hello world"

# special keys and chords (sent to the current keyboard focus):
runctl key enter                           # press Enter (e.g. submit a form)
runctl key tab
runctl key esc
runctl key cmd+a                           # select all (use ctrl+a on Windows)
runctl key ctrl+shift+t
runctl key alt+f4

runctl check                               # report permission status
runctl check --request                     # macOS: prompt for screen recording

# watch a window (or a whole screen) and notify another program on change:
runctl watch --window "微信"                                  # NDJSON to stdout
runctl watch --screen primary                                # watch the primary monitor
runctl watch --screen 1                                      # watch monitor index 1 (see `list`)
runctl watch --screen full                                   # watch the whole stitched desktop
runctl watch --window WeChat | my-notifier                   # pipe events to a consumer
runctl watch --window "微信" --background-only \             # only when not focused
  --exec 'osascript -e "display notification \"new message\""'
runctl watch --window "微信" --webhook http://127.0.0.1:8731/hook   # POST event JSON
runctl watch --window "微信" --webhook http://127.0.0.1:8731/hook --post-image  # + screenshot
runctl watch --window "微信" --duration 30                   # watch for 30s, then exit
runctl watch --window "微信" --until-change --duration 30     # wait up to 30s for a change

# machine-readable output for agents:
runctl --json list
runctl --json click --screen 1 --x 480 --y 220

Multiple monitors

The tool makes no assumption about how displays are arranged (left/right, above/below, negative offsets, mixed DPI). All geometry is read live at runtime, so the same binary works on any layout without code changes.

Run list first — it prints each monitor's index, global offset, size, and scale. The index is what you pass to --screen.

There are two ways to give coordinates:

• Global coordinates (no --screen): the OS already lays every monitor out in one virtual coordinate space, so a single global (x, y) addresses any display. A monitor to the left or above the primary has a negative offset — that's expected and supported.
• Per-monitor coordinates (--screen N): pass the pixel coordinates you read off shot --screen N, and the tool converts them with global = offset + local / scale using that monitor's live offset and scale. This is the easy path: screenshot a monitor, find your target's pixel in the PNG, click it with the same numbers.

shot --screen full stitches every monitor into the OS logical coordinate space (HiDPI captures are downscaled to logical size), so the composite is correct for any arrangement.

Coordinates and DPI

shot PNGs are in physical pixels; on a HiDPI display that's larger than the logical coordinate space the cursor uses (e.g. 2x on Retina). --screen N handles this conversion for you. If you click with global coordinates instead, divide screenshot pixels by that monitor's scale (from list) yourself. On Windows, per-monitor DPI awareness (set at startup) keeps capture and click in the same space. As a habit, run move before click to eyeball the landing spot before committing.

Watching a window or screen

watch polls a target on an interval, compares each frame to the previous one, and when enough pixels change it emits one JSON line (NDJSON) on stdout and, if --exec is given, runs a command. It runs until interrupted.

Pick exactly one target:

• --window <substr> — a single window, matched by title/app substring. It is re-resolved every tick, so closing and reopening the app does not kill the watcher.
• --screen <N|primary|full> — a whole monitor (by index from list), the primary monitor, or full (the stitched virtual desktop). --background-only has no effect here (focus is a window-only concept).

runctl watch --window "微信" \
  --interval 1000 \          # poll every second
  --threshold 0.02 \         # ignore changes smaller than 2% of pixels
  --cooldown 3000 \          # after firing, stay quiet for 3s
  --background-only \        # only fire when the window is NOT focused
  --exec './on-message.sh'   # run this on each change

runctl watch --screen primary --threshold 0.05   # watch the whole primary screen

Each event looks like (target is window:<substr> or screen:<sel>):

{"event":"change","target":"window:微信","ratio":0.087,"ts":1750000000000}

There are three ways to notify other programs. All are optional and can be combined — the simplest setup uses none of them and just reads stdout (or the exit code) directly:

• stdout (NDJSON) — always on; pipe the stream into a consumer.
• --exec <cmd> — runs via sh -c (cmd /C on Windows) with the event in environment variables RUNCTL_EVENT, RUNCTL_TARGET, RUNCTL_RATIO, RUNCTL_TS. Handlers run one at a time (the watcher blocks until it returns).
• --webhook <url> — POSTs the same event JSON (Content-Type: application/json) to an http/https URL, with a 5s timeout so a hung endpoint can't stall the watcher. A failed POST is logged and the watch continues. Add --post-image to embed the current screenshot in the payload as base64 PNG (fields image_png_b64, image_w, image_h) so the receiver can run vision on what changed. The image is only ever sent to the webhook — the stdout NDJSON stays small.

With --post-image, the webhook body looks like:

{
  "event": "change", "target": "window:微信", "ratio": 0.087, "ts": 1750000000000,
  "image_png_b64": "iVBORw0KGgo...", "image_w": 1200, "image_h": 800
}

Bounding how long it runs

By default watch runs until interrupted. Two flags bound it:

• --duration <seconds> — stop after this many seconds (a soft cap; actual runtime may exceed it by roughly one poll's capture time). Units are seconds, unlike --interval/--cooldown which are milliseconds.
• --until-change — exit as soon as the first change fires.

Combined, --until-change --duration 30 is a synchronous "wait up to 30s for a change" — ideal as an agent primitive. The exit code distinguishes the outcome:

Outcome	Exit code
A change fired (with --until-change)	0
--duration elapsed, not waiting for a single change	0
--until-change timed out with no change	124
Window not found / bad arguments	1 / 2

When bounded, the last line on stdout is a summary event:

{"event":"watch_end","target":"window:微信","events":2,"timed_out":false,"duration_s":30.0,"ok":true}

So an agent can wait for the first change with no webhook or hook at all — just branch on the exit code:

if runctl watch --window "微信" --until-change --duration 60; then
    echo "something changed"   # exit 0
else
    echo "60s with no change"  # exit 124 (or 1/2 on error)
fi

Without --duration the watch is unbounded: runctl watch --window "微信" runs until interrupted, and --until-change alone waits indefinitely for the first change and then exits.

Caveat — this detects pixel change, not meaning. It cannot tell a new message apart from you typing, scrolling, or a timestamp ticking. To make it behave like a "new message" signal: keep --background-only on (your own activity happens while the window is focused), raise --threshold to ignore small UI churn, and use --cooldown to debounce. For a precise signal, point a smaller capture at just the unread-badge area instead of the whole window (a future --region option could narrow this further).

Capturing a window requires the same Screen Recording grant as shot on macOS.

Permissions

macOS

macOS gates these capabilities behind TCC, so the binary must be granted:

• Screen Recording — required for shot (System Settings > Privacy & Security > Screen Recording)
• Accessibility — required for click, move, type (System Settings > Privacy & Security > Accessibility)

The Accessibility prompt appears the first time an input command runs; add the binary (or the terminal you launch it from) to the list and re-run. runctl check --request triggers the Screen Recording prompt.

Two things to know:

• TCC binds the grant to the binary's code signature, so recompiling can reset the grant. During development, grant your terminal (Terminal/iTerm) instead, or sign the binary with a stable identity.
• macOS Sequoia re-prompts for Screen Recording periodically (~monthly), which interrupts long-running unattended use.

Windows

No special permission is needed to synthesize input or capture the screen. The tool declares per-monitor DPI awareness at startup so capture and click share one physical-pixel coordinate space. (To drive a window running as administrator, run runctl as administrator too — UIPI blocks input from a lower-integrity process.)

Using it from an agent

Each command is a single process call with structured --json output and a clean exit code, which maps directly onto a tool-call interface (e.g. Runify's run_command / run_skill): the agent runs one command, reads the JSON result, and decides the next step.

A full "watch a chat, then reply" loop is just a sequence of these calls. Pin the exact window once with windows, then drive everything by its id so a second window of the same app can never be picked by mistake:

runctl --json windows                            # 0. find the WeChat main window's id, e.g. 51234
runctl watch --window-id 51234 --until-change    # 1. block until a message arrives
runctl --json shot --window-id 51234 --out msg.png --grid   # 2. capture it (grid helps read coords)
# 3. the model reads msg.png and decides the reply + the input-box pixel (e.g. 440,600)
runctl activate --window-id 51234                # 4. bring WeChat to the front
runctl click --window-id 51234 --x 440 --y 600 --shot-w 1280 --shot-h 720   # 5. focus the input box
runctl type "你好,稍后回复"                       # 6. type the reply
runctl key enter                                 # 7. send

Every window-targeting command takes either --window <substr> (convenient) or --window-id <n> (exact, from windows). For click/move/drag/scroll, pass --shot-w/--shot-h (the screenshot size from step 2's --json) and the pixel is mapped proportionally onto the window's live rectangle — correct at any DPI/scale, no manual math. (Omit --shot-w/--shot-h and x/y are taken as window-relative points with no scaling.) activate (step 4) matters because type goes to whatever is focused: raise WeChat and click into its input box first, or the text lands elsewhere.

Add --grid to step 2's shot to overlay a labelled coordinate grid on the image. The numbers it prints are in the screenshot's own pixel space — i.e. exactly the --x/--y values to pass back (with --shot-w/--shot-h = that image's size), so the model can read a target's coordinate straight off the picture. If edge labels alone aren't precise enough, add --grid-coords (with a coarse --grid-step, e.g. 200) to label every intersection with its x,y, putting a reference value right next to the target.

A process that can screenshot the whole display and click anywhere has a very large authority surface. If runctl runs inside an agent execution environment, constrain when it can capture and where it can click rather than handing the agent unrestricted control of the machine.

License

MIT — see LICENSE.