Releases: TestSprite/testsprite-cli
Releases · TestSprite/testsprite-cli
Release list
v0.4.0
[0.4.0] - 2026-07-16
Added
test cancel <run-id...>— user-initiated cancel of in-flight runs (the real stop button; Ctrl-C only detaches). A single id renders the run card with statuscancelled(plus an advisory when it was already cancelled); multiple ids print a{cancelled, alreadyCancelled, conflicts, notFound}summary. Exit codes: 4 when any id is not found, else 6 on conflicts, else 0.--dry-runsupported.- Graceful Ctrl-C during
--wait— SIGINT/SIGTERM now detaches cleanly instead of killing the process mid-poll: the in-flight request aborts immediately, stdout gets the same partial{runId, status: "running"}envelope as the request-timeout path, and stderr states the truth — the server-side run keeps executing (and billing) — with a re-attach hint and atest cancelpointer. Exit 130/143/129 per the documented signal contract; a second signal forces a hard exit. Interrupting never cancels the server-side run — that's whattest cancelis for. project delete <project-id> --confirm— permanently delete a project and everything under it (its frontend/backend sub-projects, all their tests, and backend fixtures), mirroring the Portal's cascade delete. Requires--confirm(the CLI never prompts);--dry-runpreviews the response shape without a network call. Standard exit codes: 0 success, 3 auth, 4 not-found (or already-deleted), 5 validation (e.g. missing--confirm).- Backend stdout and traceback in results —
test resultand the failure bundle now surface the backend test's captured stdout and Python traceback: full content in--output json(and inresult.json/failure.jsonbundle files), and a bounded 20-line tail with a byte count in text mode. No change for frontend tests, passing runs, or older backends. - Backend dependency declarations are now readable and editable —
test getsurfacesproduces/consumes/category, andtest updateaccepts--produces/--needs/--category(previously create-only). - Version-compatibility handshake — the CLI reads the backend's advertised minimum-supported-version on every response and prints a one-line upgrade advisory on stderr when the running version is below the floor (honors the same opt-outs as the update notice; never alters exit status). A
CLIENT_TOO_OLDrejection (HTTP 426) is now a first-class error: exit 14, non-retriable, rendered with upgrade guidance and the version gap. - V3 routing visibility —
auth statusanddoctorrender arouting: v2|v3line when the backend reports the account's routing, and V3-routed accounts get one consolidated advisory listing the known V3-path behavior gaps. Text mode only — JSON consumers readv3Enabledfrom the/mepayload; absent-safe against older backends.
Changed
- The
testsprite-verifyagent skill routes local-only changes to the TestSprite MCP — the skill now states the reachability gate explicitly: the CLI verifies reachable deployed URLs only; when the change is only running locally, the skill hands off to the TestSprite MCP when available (an explicitly named tool always wins), instead of failing against localhost.
Fixed
project create --descriptionnow fails fast with a clear validation error — projects have no description field, so the flag's value was previously dropped silently; the error points at test-level descriptions (test create --description) instead.- Standalone backend run cards no longer show a misleading step summary —
test run/test wait/test reruncards for backend tests rendersteps: n/a (backend)instead of0/0 (passed=0, failed=0)(backend tests have no per-step storage).
v0.3.0
[0.3.0] - 2026-07-09
Added
testsprite doctor— one-command environment diagnostic that checks your Node version, credentials, endpoint reachability, and installed agent skills, and reports what's misconfigured.test scaffold— emit a schema-correct starter plan (frontend) or a backend test skeleton to bootstrap a new test without hand-writing the JSON.test lint— offline validator for plan / steps files; catches malformed test definitions before they are sent to the server.test diff <run-a> <run-b>— compare two runs of the same test to isolate what changed between a passing and a failing run.test flaky <test-id>— repeat-run flaky-test detector. Replays a test N times (--runs, default 5), aggregates the outcomes, and reports a stability verdict (stable/flaky/failing) plus therunIdandfailureKindof every attempt that did not pass. Replays run with auto-heal off (strict verbatim) so a nondeterministic pass/fail can't be masked. Exit code is 0 only when every attempt passed, so CI can gate a merge on flakiness. Flags:--runs(1–10),--until-fail,--timeout,--output json.- JUnit XML report export for batch runs.
test run --alland batchtest rerunaccept--report junit --report-file <path>to write a CI-friendly XML sidecar after--waitpolling completes. The report is written even when the batch exits non-zero;--output jsonis unchanged;--dry-runwrites a canned sample without network calls. test waitis now variadic — pass several run ids to attach to and poll multiple runs in a single invocation.agent status— report which TestSprite skills are installed for each agent target and whether they are current; installed skills are now stamped with a version/hash marker.- New
agent installtargets: GitHub Copilot, Windsurf, and Kiro (experimental), alongside the existing Claude / Cursor / Cline / Codex / Antigravity targets. project credential/project auto-auth— configure a project's backend credentials (static credential, free) or a recurring auto-auth token (Pro) from the CLI, with surfaced auth warnings and managed-credential guidance.- Proxy support — the CLI now honors
HTTPS_PROXY/HTTP_PROXY/NO_PROXYfor use behind corporate and CI proxies. NO_COLORsupport — colored output is suppressed whenNO_COLORis set, per no-color.org.- "New version available" notice — a non-blocking, 24h-cached npm version check prints an upgrade hint on stderr. Opt out with the documented env var; automatically silenced in CI and under
--output json.
Changed
- Node.js 20.19+, 22.13+, or 24+ is now the minimum supported runtime. The CLI checks the running Node version at startup and exits with a clear message on an unsupported version; builds and CI run against Node 20 and 22.
- Graceful shutdown — the CLI handles termination signals cleanly and guards against broken-pipe (
EPIPE) errors when its output is piped to a closing consumer (e.g.| head). - Interactive prompts and preamble now go to stderr, keeping stdout pure for machine consumers even in interactive mode.
- Empty environment variables are treated as unset when resolving config, so
TESTSPRITE_API_URL=no longer overrides the built-in default with an empty string. agent installdefaults--targettoclaudein non-interactive / CI contexts (matchingsetup).- The
usagecommand no longer implies backend test runs are free. setup's "Next steps" guidance no longer suggeststest listbefore any project exists.
Fixed
- Timeouts & polling:
RequestTimeoutErroris now classified as a timeout in the--all --waitfan-out; per-attempt timeout timers are cleared so they can't fire late;run --all --waitno longer polls still-queued runs past the shared deadline; a partial result is emitted on stdout whenrun --wait/test waittimes out (so a redirected file is never zero-byte). - Batch rerun: the exit code is preserved and auth errors escalate correctly; explicit ids combined with
--all— or--status/--skip-terminalwithout--all— are rejected with a clear validation error; auto-minted idempotency keys are surfaced under--output json. - HTTP: non-JSON
200responses map to a typed error envelope instead of crashing the parser. - Failure bundles / artifacts: artifact downloads retry on transient errors and guard the default run-id path; the
--outdirectory no longer sweeps unrelated pre-existing files (data-loss fix); run-scoped per-step error text and step type are surfaced. - Input validation (fail fast, before any network call): malformed API keys, invalid
--request-timeout, directory--code-file/--outpaths, blank or whitespace-only--name(test and project create/update), blank inline project passwords, fractional pagination flags / page sizes, and--sinceoverflow are all rejected up front withVALIDATION_ERRORrather than crashing or failing late server-side.--outputis validated uniformly across all command groups. - Setup / auth: the endpoint is validated before the key check; the typed API-error envelope is preserved when key verification fails; the per-request timeout is honored during
configure. - Misc: cursor pagination no longer drops empty pages; trailing-dot hostnames are treated as loopback by the local-target guard; buffered input is preserved between interactive prompts; the Codex managed-section skill check requires a complete section;
code getstrips a leading BOM and rejects an empty--out.
Security
- INI injection: CR/LF characters are stripped from credential values before they are written to
~/.testsprite/credentials. - Symlink fail-close: the own-file
agent installpath applies its symlink containment guard under--dry-runas well, so a planted symlink cannot place or clobber files outside--dir.
v0.2.0
[0.2.0] - 2026-06-29
Added
- Seed-suite onboarding skill.
agent installnow installs a second skill by default —testsprite-onboard— which guides your coding agent to create a first test suite in a repository that doesn't have one yet (alongside the existingtestsprite-verifyskill). Useagent install --skill <name>to install only a specific subset.
Changed
test resultnow reports the test verdict and execution status as separate fields. The latest-result output gains averdictfield (the run's pass / fail / blocked judgement) and anexecutionStatusfield (how the run terminated);summaryis now a human-readable string.- Breaking change to
--output json:summarywas previously an object ({ passed, failed, skipped }). Scripts that readsummary.passed/.failed/.skippedmust move to the newverdict/executionStatusfields. The legacystatusfield is unchanged.
- Breaking change to
project create,project update, andtest run --allnow print the[dry-run] sample response — not from the serverbanner under--dry-run, consistent with every other command.- The CLI no longer suggests TypeScript/JavaScript test code is supported — TestSprite runs all test code as Python.
Fixed
- Security (failure-bundle writer):
test failure getandtest artifact getnow validate the response's step index and evidence kind before composing file paths, so a malformed or hostile API response can no longer write files outside the chosen--outdirectory. test create --type backendandtest code putnow reject a non-Python--code-fileimmediately with a clear validation error, instead of failing late server-side.test failure get --outis validated before the network call;test code get --outnow writes atomically (no truncation if the fetch fails); artifact downloads retry on transient transport errors.- Clearer validation messages for malformed
--endpoint-urlvalues and profile names; argument-parse errors now emit a structured JSON envelope under--output json. - Batch rerun dispatch is de-duplicated and serialized.
v0.1.2
v0.1.2
testsprite setup— onboarding is consolidated into one command (formerlyinit). The granular commands stay as hidden, deprecated aliases (auth configure→setup,auth whoami→auth status,auth logout→auth remove).- The CLI now reports its version in the
User-Agentheader. - README: the launch video no longer renders as a bare URL on the npm page.
v0.1.1 — README launch-video patch
Docs-only patch release: the npm 0.1.0 package was published before the README launch video was updated, so the npm page embedded the old video. npm READMEs are immutable per version — this release refreshes it. No code changes; the package file list is identical to 0.1.0.
Changed
- README: point the launch video at the updated public asset
- Prettier-clean the README video block (fixes
format:checkonmain)
Note: this version was published to npm manually (
npm publishfrom a maintainer machine, same as0.1.0). Thereleaseworkflow run attached to this tag is expected to fail at the publish step — the repo has noNPM_TOKENsecret configured yet. Full changelog:CHANGELOG.md.