Skip to content

Gate execution reports 'passed' when ruff/mypy aren't installed in the gate env #13

Description

@Svaag

The launch-proof dogfood (hyrule-cloud#28 → PR #29) exposed a gate-fidelity gap: the loop's gate_execution node reported passed, but the diff actually failed both ruff (unused import) and mypy (2 type errors) when run the way the repo's CI does.

Root cause

Target repos expose their linters via an extra/group, not the base venv (hyrule-cloud: uv run --extra dev ruff/mypy ...; CI uses --extra dev). The loop's gate run invokes the bare tool, so ruff/mypy are not found and the gate effectively only runs what's importable — yielding a false "passed". A coding agent then opens a draft PR that fails the real branch-protection checks (caught only by human review).

Ask

  1. Run gate commands with the target repo's dev dependencies available (e.g. resolve and use --extra dev / the repo's dependency group), so ruff/mypy/pytest actually execute.
  2. Treat tool-not-found / non-zero spawn as a gate failure, never a pass — a gate that cannot run is not a gate that passed.
  3. (Related) surface the gate command output in the run trace so false passes are visible.

Evidence

PR #29 reported gates green from the loop; locally uv run --extra dev ruff check . + mypy found 3 issues (1 unused import, 2 objectstr coercions). Fixed by hand to land the PR.

Metadata

Metadata

Assignees

No one assigned

    Labels

    agentic-ispAS215932/Hyrule agentic ISP operating-loop work

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions