ClaudeCliBackend on Windows: bare "claude" not resolvable (npm .cmd shim), FileNotFoundError swallowed, whole run silently scores 0.0

[hiz-eng]

Summary

On a Windows machine where Claude Code is installed via npm, ClaudeCliBackend never manages to invoke the CLI at all, and the failure is completely silent: every task gets an empty answer, the experiment "completes" in a few seconds, and all scores come back 0.0. Nothing in the output tells the user the CLI was never executed.

Two stacked problems:

1. Path resolution. ClaudeCliBackend defaults to claude_path="claude" and calls subprocess.run([self.claude_path, ...]). An npm install of Claude Code on Windows puts only shims on PATH (claude.cmd, claude.ps1, and a sh script named claude); there is no claude.exe on PATH. Python's CreateProcess does not apply PATHEXT to bare names, so subprocess.run(["claude", ...]) raises FileNotFoundError: [WinError 2].

2. Silent swallow. In _call the subprocess call is wrapped in except Exception: return "" (backend.py, skillopt 0.2.0, around line 623), and attempt_with_tools does the same (except Exception: resp = ""). Because the exception path returns before _detect_cli_error runs, last_call_error is never set and no warning is logged. The empty string is then treated as a legitimate model answer, so the run proceeds and scores 0.0 everywhere. This is the same failure class that fix(skillopt-sleep): surface codex auth/model/version failures instead of silently scoring 0 #92 fixed for the codex backend, and the same "broken setup looks like nothing to optimize" trap the _CLI_ERROR_MARKERS comment (issue ClaudeCliBackend hardcodes --bare, breaking subscription-token auth (silent 0-score) #68) warns about, but the claude path still has it for process-spawn failures.

Environment

• Windows 11 Pro (10.0.26200), Python 3.14.3, skillopt 0.2.0 from PyPI
• Claude Code CLI 2.1.206 installed via npm (%APPDATA%\npm\claude.cmd shim wrapping %APPDATA%\npm\node_modules\@anthropic-ai\claude-code\bin\claude.exe)
• ANTHROPIC_API_KEY set (so the --bare branch is taken; not relevant to the bug)

Repro

import subprocess
subprocess.run(["claude", "--version"], capture_output=True, text=True, timeout=60)
# FileNotFoundError: [WinError 2] The system cannot find the file specified

Then any run that routes through ClaudeCliBackend (e.g. python -m skillopt_sleep.experiments.run_experiment --persona researcher --backend claude --model claude-sonnet-5) finishes in seconds with baseline_holdout 0.0, after_holdout 0.0, zero warnings. Observed: 12 tasks, 4 nights, 3.8 s wall time, all scores 0.0.

Expected

Either the CLI is found (resolve the shim), or the run fails loudly ("claude CLI could not be executed: [WinError 2] ...") instead of reporting a clean 0.0 result.

Confirmed workaround

Passing the real exe path makes everything work end to end:

be = ClaudeCliBackend(model="claude-sonnet-5",
                      claude_path=r"C:\Users\<user>\AppData\Roaming\npm\node_modules\@anthropic-ai\claude-code\bin\claude.exe")

With that one change the same 12-task researcher run passes (holdout 0.3056 to 0.8611).

Suggested fix

• Resolve claude_path with shutil.which("claude") at init (which honors PATHEXT and finds claude.cmd), or probe for the exe behind the npm shim. PR fix: make ClaudeCliBackend work on Windows (.cmd shim resolution + argv length) #98 already implements shim resolution and looks like it would fix problem 1; linking it here since it has no tracking issue.
• Independently of resolution: in _call and attempt_with_tools, catch the spawn failure explicitly, set last_call_error, and log a warning (mirroring what fix(skillopt-sleep): surface codex auth/model/version failures instead of silently scoring 0 #92 did for codex) so a dead CLI can never masquerade as a 0.0-scoring model.

Happy to provide more details from the affected machine if useful.