ci: make the Tests workflow actually run, and gate the deploy-blockers

[dgokeeffe]

CI has never run the test suite on this repo

The Tests workflow has exactly one run in its history — a manual workflow_dispatch in May — and that run has zero jobs. Dependency Audit and Update Lockfile have never produced a run at all.

Cause: all three target databrickslabs-protected-runner-group. That group is granted per-repo across the org (ucx, dqx, dlt-meta use it successfully) and this repo was never granted it, so jobs targeting it are never scheduled — silently, with nothing anywhere to notice.

What that already cost

• chore(deps): bump pydantic-core from 2.46.4 to 2.47.0 #84 bumped pydantic-core to 2.47.0 while pydantic==2.13.4 pins pydantic-core==2.46.4 exactly. requirements.txt became unsatisfiable, so every deploy from main failed until fix(deps): repair unsatisfiable requirements.txt + clear 19 of 20 Dependabot alerts #110 repaired it. GitHub reported it MERGEABLE; nothing installed requirements.txt.
• chore(deps): bump mcp from 1.29.0 to 2.0.0 #115 and chore(deps): bump opentelemetry-sdk from 1.41.1 to 1.44.0 #116 were the same failure, caught only by resolving them by hand.
• requirements.lock had drifted from requirements.txt, because Update Lockfile never ran. Regenerated here.

Runners

test.yml, dependency-audit.yml, update-lockfile.yml → ubuntu-latest. None need org secrets (first two read-only; the third uses only the repo-scoped GITHUB_TOKEN), and hosted runners are free for public repos.

release.yml is left on the protected group — it signs release artifacts, so it deserves a deliberate decision rather than a drive-by change. ⚠️ That means releases stay blocked until the repo is granted the runner group. Worth requesting from an org admin either way.

New gates, each tied to a failure that actually happened

Gate	Catches
requirements.txt must be installable	#84 / #115 / #116
requirements.lock must match requirements.txt	the drift above
13 app modules must import	boot-time breakage with no unit test
Floor on tests that actually ran	#106 — a suite reporting "465 skipped" passes vacuously

Two bugs found while verifying this locally

1. tests/e2e/conftest.py raised an uncaught FileNotFoundError when the databricks CLI is absent — surfacing as a pytest INTERNALERROR that kills the entire session. This workflow would have failed on its first run, since hosted runners don't have the CLI. Now treated as "prerequisite absent" → skip.

2. tests/integration is excluded from the PR gate. It gates on Docker — and Docker is present on ubuntu-latest, so it would really run: building an apps-like image and executing the full setup pipeline (npm installs, CLI downloads). Confirmed locally that including it produces failures. Still runnable on demand.

Verification

Every step run locally, including with PATH stripped of the databricks CLI to simulate a hosted runner: 535 passed, 3 skipped.

Then each gate was confirmed to fail when its bug is reintroduced — the #84 pin, a truncated lockfile, and the #106 whole-session skip:

### Does the install gate catch the #84 breakage?   ✓ gate CATCHES it
### Does the lock gate catch drift?                ✓ gate CATCHES it
### Does the vacuous-suite guard catch #106?       ✓ gate CATCHES it  (parsed passed=0)

This PR should be the first thing on this repo whose CI actually goes green — which is itself the test.

[dgokeeffe]

Correction: the runner group was not the (only) cause

I claimed this PR would be the first thing here whose CI goes green. It isn't, and the diagnosis in the description is incomplete. Evidence after pushing:

# workflow_dispatch on this branch, with runs-on: ubuntu-latest
run 30997809499  status=queued  conclusion=null  jobs=0     (still queued after 3+ min)
# the original May run, on the protected group
run 25289678794  status=queued  conclusion=null  jobs=0     (queued since 2026-05-03)

So a run is created but no job is ever scheduled, and switching to ubuntu-latest does not change that.

It is not that hosted runners are unavailable to the repo. A GitHub-managed workflow in this very repo runs jobs on ubuntu-latest fine:

{"name": "update-pip-graph", "conclusion": "success",
 "labels": ["ubuntu-latest"], "runner_group_name": "GitHub Actions"}

The difference is that those runs have event: dynamic — they're injected by GitHub (Dependency Graph, Dependabot Updates), not defined in .github/workflows/. Repository-defined workflows are the ones that can't schedule a job. That points at an Actions policy on the repo or org, which I can't read (/actions/permissions → 403, needs admin) or change.

What this PR is and isn't

Still worth merging as-is — it's necessary but not sufficient:

• The tests/e2e/conftest.py FileNotFoundError → INTERNALERROR fix is a real bug fix that bites any machine without the databricks CLI, CI or not.
• requirements.lock was genuinely stale and is regenerated.
• It stops targeting a runner group this repo was never granted.
• All four gates are verified locally, including against reintroduced versions of the chore(deps): bump pydantic-core from 2.46.4 to 2.47.0 #84, lockfile-drift and fix(tests): scope e2e prerequisite skip to the e2e directory #106 bugs. They start working the moment jobs can schedule.

It does not make CI run. Nobody should read a merge here as "CI is fixed".

Needed from someone with admin

1. Repo → Settings → Actions → General — check Actions permissions (is it "Allow all actions", or a restrictive allow-list that blocks actions/checkout etc.?) and the runner policy.
2. Org → Settings → Actions → Runner groups — whether this repo is granted databrickslabs-protected-runner-group (it evidently isn't; ucx/dqx/dlt-meta have it).
3. Whichever is the org's intent, one of: grant the runner group (then revert the runs-on changes here), or permit hosted runners for repository workflows.

Note release.yml is still on the protected group, so releases are blocked too — worth resolving in the same request.