feat(host): generic app.yaml via resources, teammate host-share, jq, nested wheels - #121
Merged
Conversation
Replace the commented-out OMNIGENTS_* literals with active valueFrom resource references so app.yaml stays generic AND git-deploy-friendly (no per-deploy file edit). The host becomes resource-gated: ON when the omnigent-wheels (UC Volume) + omnigent-server-url (Secret) resources are attached, OFF otherwise (an unresolved valueFrom yields an empty string and omnigents_host_enabled() returns False). - OMNIGENTS_SERVER_URL -> valueFrom: omnigent-server-url (Secret) - OMNIGENTS_WHEEL_SPEC -> valueFrom: omnigent-wheels (UC Volume) - OMNIGENTS_FORCE_REINSTALL + ENABLE_SP_APIKEYHELPER now active (were commented out alongside the literals). Add attach_omnigent_resources.sh + make attach-omnigent-resources target to attach the two resources per app, idempotently, MERGING with existing resources (apps update --resources replaces, so merge avoids clobbering workshop challenge-repo-token). Run after grant_omnigent_host.sh. Makefile: GRANT_YAML var + OMNIGENT_SERVER_URL/WHEEL_VOLUME derivation + OMNIGENT_SECRET_SCOPE/KEY defaults for the new target.
…d wiping git_repository apps update --json is a full-body write that CLEARS unset fields — it wiped the coda app's git_repository on a live git-linked app, breaking redeploys (learned during go-live validation). The CLI's create-update --json body shape is finicky (rejects 'resources' as an unknown field when update_mask is embedded). Switch the write step to the Apps SDK directly: create_update(app, update_mask='resources', app=App(resources=[...])) — a targeted field-mask patch that touches ONLY resources, preserving git_repository and other app fields. Validated against the live coda app (git_repository + both omnigent resources present after re-run).
The share endpoint previously granted only to the calling user (the owner), so you could self-share but not share to a teammate. Add an optional grant_user in the request body; when present, share to that user instead. The owner-gate is unchanged (only the app owner may invoke the endpoint, since it acts with the SP's authority). Default behavior (no grant_user) is unchanged: share to the calling user, so the browser auto-share path still works.
_materialize_spec listed only the top level of the UC Volume, so it raised 'no .whl in UC Volume' when the omnigent build publishes wheels under wheels/<version>/ instead of flat at the volume root. Recurse (bounded depth) when the root has no wheels, pick the directory holding the newest-uploaded main omnigent- wheel, and install every wheel from that dir. Flat top-level layout still works unchanged. Tests cover flat, nested-newest-wins, and the no-wheels error.
Omnigent's native pi/claude/codex harnesses resolve their AI-Gateway bearer
via an auth command ending in `... --output json | jq -r '.access_token'`
(omnigent.inner.codex_executor._databricks_codex_auth_command). Databricks
Apps containers ship no jq, so the pipe yields an EMPTY token and the harness
fails with "Failed to resolve API key" — the reported pi-native breakage.
Add install_jq.sh (static binary to ~/.local/bin, same fetch pattern as
install_tmux.sh) and wire it into both paths: _run_step("jq") in app.py
(interactive) and _ensure_jq() in omnigents_host.py before _run_setup_once
(host). Idempotent; non-fatal on failure.
opencode tool calls 400'd with "Unknown name exclusiveMinimum ... Cannot find field" on the mlflow chat-completions gateway surface, which validates tool schemas against an OpenAPI-3.0 subset (shared with the Gemini translator). exclusiveMinimum/Maximum are NUMBERS in JSON Schema draft-2020 but boolean modifiers in OpenAPI 3.0, so the parser rejects the whole request. Add exclusiveMinimum/Maximum, multipleOf, minLength/maxLength/pattern, minItems/maxItems/uniqueItems, and minProperties/maxProperties/patternProperties to GEMINI_UNSUPPORTED_SCHEMA_KEYS. strip_unsupported_schema_keys recurses so they're removed at any depth (incl. nested properties[N].value). These are advisory constraints — dropping them keeps the tool callable; Claude/GPT ignore them. Regression test added.
Captures the ucode-duplication findings from the 2026-07-20 debug session: CoDA's setup_*.py reimplements ucode's per-agent config writers (the root of the exclusiveMinimum dialect bug), ucode's DATABRICKS_BEARER seam as the clean auth integration, and — critically — that ucode does NOT cover the Omnigent runner native-harness path (which is where the reported errors actually came from). Proposes migrating only the interactive path; keeps the shipped hotfixes.
The base app.yaml targeted an Azure workspace (adb-7405614666872455) with env-specific Otel/MLflow/challenge-repo config. The active deploy target is now the aws-syd workspace (coda app, dbc-d8b32eb0-866b, AWS ap-southeast-2): point DATABRICKS_GATEWAY_HOST there, GEMINI_MODEL=databricks-gemini-3-pro, ENABLE_SP_APIKEYHELPER=true, MAX_CONCURRENT_SESSIONS=5. Parity with the live deployed coda config (verified via workspace export). Retains app.yaml.aws-syd as the untracked source overlay.
pi (and any omnigent native harness) showed only its single hard-coded default
model (databricks-claude-sonnet-4-6) instead of the workspace's full endpoint
list. Root cause: omnigent's pi_native_credentials resolves the model catalog
via resolve_databricks_workspace("omnigents-host") →
Config(profile="omnigents-host").authenticate(), but that profile is
host-only (credentials are brokered over loopback, never persisted). The SDK
raised "cannot configure default credentials", the catalog fetch failed, and
pi fell back to one model.
Two coordinated changes so the SDK can mint via the existing broker shim:
- token_helper.write_databricks_token_wrapper: emit the FULL OAuth shape the
SDK's DatabricksCliTokenSource requires (access_token + token_type + expiry),
not a bare {access_token}. Broker gives no expiry and always mints fresh, so
set a short now+5min expiry (well inside the ~1h SP TTL) → SDK re-invokes the
shim for a fresh token rather than caching one of unknown lifetime.
- omnigents_host._write_oauth_profile: write auth_type = databricks-cli so
Config(profile=...).authenticate() runs `databricks auth token --profile
omnigents-host`, which resolves to the broker shim (first on the runner PATH).
Verified live: broker token → GET /api/2.0/serving-endpoints = HTTP 200,
50 endpoints (11 claude, 9 gemini, 19 gpt). Tests updated for the new shim
shape + profile auth_type.
…SON always Follow-up to 1fc58d0 — that made the profile 'authable' in theory but two live failures on the AWS container (sdk 0.106.0) proved it insufficient: 1. The SDK's DatabricksCliTokenSource resolves the `databricks` binary via its OWN lookup, NOT $PATH — so it ran the REAL CLI in ~/.local/bin (no OAuth cache) → 'databricks OAuth is not configured'. Fix: write `databricks_cli_path = <broker shim>` into the omnigents-host profile. It's a normal Config attribute the SDK reads straight from .databrickscfg, so it reaches the in-runner catalog fetch WITHOUT needing omnigent's host→runner env allowlist (which we don't own / would need a wheel rebuild to change). 2. The SDK invokes `auth token --profile <p>` WITHOUT `--output json` yet json.loads()s stdout (verified in the container's 0.106.0 source). The shim only emitted JSON when --output json was present, so the no-flag path printed a raw token → 'cannot unmarshal CLI result: line 1 column 1'. Fix: shim now emits the access_token/token_type/expiry JSON unconditionally (matches the real CLI, which defaults auth token to JSON). Verified against the live container: broker token → serving-endpoints = 200, 50 models; shim --output json returns a valid JWT; DATABRICKS_CLI_PATH changes the SDK's error from 'OAuth not configured' to 'cannot unmarshal' (proving it now runs the shim). Tests updated.
…nested wheels
Brings forward feat/omnigent-host-followups (13 commits, 2026-07-20) — the last
branch in the repo with unlanded work. It never had a PR. Rebuilt against
current main rather than merged, because its app.yaml is a different personal
workspace (see below).
## Generic app.yaml (docs/agent-instructions.md §5)
OMNIGENTS_SERVER_URL and OMNIGENTS_WHEEL_SPEC become `valueFrom` resource
references instead of hardcoded values. §5 says these must be commented out or
defaulted off before landing on main; main currently carries one workspace's
Omnigent server URL and UC Volume path.
The on/off switch becomes "are the resources attached" rather than "did you edit
this committed file": an unresolved valueFrom yields an empty string and
omnigents_host_enabled() returns False. Adds attach_omnigent_resources.sh and a
`make attach-omnigent-resources` target to attach them, using SDK
create_update with an update_mask so attaching doesn't wipe git_repository.
**Only those two vars changed.** The branch's own app.yaml was its author's
aws-syd workspace — taking it wholesale would have swapped one set of personal
values for another (a different DATABRICKS_GATEWAY_HOST), flipped ENABLE_CODEX
and ENABLE_GEMINI back on against main's comments explaining why they're off,
dropped MLFLOW_TRACING_ENABLED / CLAUDE_CODE_OTEL_ENABLED / PROXY_TRACE_CONTENT
/ CLAUDE_INSTALL_METHOD, and removed CODA_DISABLE_OWNER_CHECK. Verified the env
var set is otherwise byte-identical to main.
CHALLENGE_REPO_URL still holds a personal sandbox URL on main. Left alone here
to keep this reviewable; worth the same treatment separately.
## Teammate host-share
POST /api/omnigent-host/share accepts an optional `grant_user` in the body, so
the owner can share the SP-owned host with a teammate. Previously it could only
ever self-share. This is the gap §5 describes: "An SP-owned host is invisible in
a human's personal picker unless shared via PUT
/v1/hosts/{id}/permissions/{user_id}". The endpoint stays owner-gated by the
before_request check — only the owner can issue a grant.
## SDK-authable broker shim
The CLI shim now emits the full access_token/token_type/expiry JSON that
databricks-sdk's DatabricksCliTokenSource requires, unconditionally rather than
only under `--output json`. The SDK builds the command without that flag yet
still json.loads() the output, because the real CLI defaults `auth token` to
JSON; the old raw-token path made Config(profile=...).authenticate() fail with
"cannot unmarshal CLI result", so pi fell back to a single-model picker. Expiry
is deliberately short (5 min) so the SDK re-invokes the shim for a fresh token
rather than caching one whose real lifetime we can't prove. Checked that no
caller depends on the old raw output — provision_coda_pats.sh already passes
--output json, and nothing uses --output text.
## jq, and nested wheel layouts
Omnigent's native harnesses resolve their bearer via a command ending in
`| jq -r .access_token`; there's no jq in the Apps image, so without it the
harness reports "Failed to resolve API key". Adds install_jq.sh.
Host wheels are now resolved from `wheels/<version>/` subdirectories, picking
the directory holding the newest-uploaded main wheel so a stale version
alongside a new one isn't mixed in.
## Fixes made while integrating
- The jq setup step wasn't registered in setup_state["steps"]. _update_step()
silently no-ops on an unknown id, so the step would have run invisibly — no
progress line, and no error surfaced if it failed.
- install_jq.sh ignored GITHUB_RELEASE_MIRROR, which install_gh.sh,
install_micro.sh and install_databricks_cli.sh all honour. A firewalled
deploy would have failed to reach github.com. Now mirrored, downloads to a
temp path, verifies the binary actually runs before installing it as `jq`
(a truncated download or HTML error page would otherwise land on PATH and
silently produce an empty token — the exact failure the script exists to
prevent), and is best-effort: a download failure warns and exits 0 rather
than marking the whole app setup errored, since jq only matters on the
Omnigent path, which is off by default.
- The branch's Gemini schema-key list is a superset of what I landed in #119
(it also strips minLength/maxLength/pattern, minItems/maxItems,
minProperties/maxProperties/patternProperties, with the clearer rationale
that Gemini's OpenAPI-3.0 parser has no draft-2020 fields). Took the superset
and kept both sets of tests.
## Verification
541 passed, 3 skipped (up from 536). requirements install + lockfile + module
imports all pass. app.yaml env var set diffed against main to confirm only the
two intended entries changed. install_jq.sh exercised with jq absent and a
broken mirror: warns, exits 0, installs nothing.
dgokeeffe
added a commit
that referenced
this pull request
Aug 5, 2026
…ml files (#122) docs/agent-instructions.md §5 requires personal/workspace values to be commented out or defaulted off before landing on main. This is a public repo and several had accumulated: a customer workspace id, a UC catalog, a self-hosted MLflow app URL, and a personal sandbox repo. Each is both an information leak and actively wrong for anyone else deploying the template. Removed (commented out with placeholders, or already resolved via valueFrom): app.yaml DATABRICKS_GATEWAY_HOST adb-7405614666872455…azuredatabricks.net CLAUDE_CODE_OTEL_CATALOG_SCHEMA edp_aisandbox_aisandbox_dev.ppcs MLFLOW_OSS_URL coda-mlflow-oss-7405614666872455… CHALLENGE_REPO_URL github.com/david-okeeffe_data/dok-sandbox app.yaml.workshop the same four, plus OMNIGENTS_SERVER_URL / OMNIGENTS_WHEEL_SPEC switched to the valueFrom resources the base app.yaml already uses (#121) app.yaml.template DATABRICKS_GATEWAY_HOST — see below ## DATABRICKS_GATEWAY_HOST has three states, not two Unset is the right default and is why this one is commented rather than blanked: unset -> derive from the workspace id (or an Azure DATABRICKS_HOST) and PROBE for reachability (utils.get_gateway_host tier 2) set URL -> TRUSTED, no probe (tier 1) set "" -> explicitly DISABLE the gateway, use serving-endpoints So `value: ""` is not "no value" — it turns the gateway off. And because a set URL is trusted without probing, a stale one doesn't fall back: a token minted at one workspace is invalid at another's gateway, so every model call returns "400 Invalid Token". That last point is why the new guard caught a real bug in app.yaml.template, which shipped DATABRICKS_GATEWAY_HOST *uncommented* with the placeholder still in it. Deploying the template unedited would pin the gateway to the literal string `https://<your-gateway-id>.ai-gateway.<env>.cloud.databricks.com` and fail every model call, where leaving it out would have worked. Now commented, with the trap spelled out. ## Safety of each removal - CLAUDE_CODE_OTEL_CATALOG_SCHEMA: claude_otel.py returns False when it's absent, so OTEL is simply off. CLAUDE_CODE_OTEL_ENABLED also flipped to "false" so the flag doesn't advertise a feature that can't run. - MLFLOW_OSS_URL: proxy_tracing._enabled() already requires both the flag and a non-empty URL; the flag was already "false". - CHALLENGE_REPO_URL: app.py only registers the "challenge" setup step when it's set, and install_challenge_repo.sh exits 0 when unset. Commented together with CHALLENGE_REPO_READ_TOKEN, since an unresolved valueFrom for it is the source of the known "error resolving resource challenge-repo-token" boot warning. ## Guard tests/test_app_yaml_overlays.py grows two checks over every tracked app.yaml*: no active env value may contain a workspace id, a concrete UC Volume/catalog, an AWS workspace host or a personal GitHub repo; and DATABRICKS_GATEWAY_HOST must not be pinned. Commented-out examples are exempt — that's where placeholders belong. Verified the guard fails when the UC catalog value is reintroduced. 549 passed, 3 skipped. All four app.yaml* files still parse and still declare every ENABLE_<CLI> toggle.
dgokeeffe
pushed a commit
that referenced
this pull request
Aug 5, 2026
….17.0) Closes #118, #120, #121, #122 - Session creation prompt: ask users to reuse existing sessions before creating new - MAX_CONCURRENT_SESSIONS backend cap (env var, default 5) with TOCTOU-safe check - Session count label in tab bar with updates on all create/close/exit paths - xterm.js ClipboardAddon for OSC 52 (copy-paste inside Claude Code) - Write batching with requestAnimationFrame to prevent escape sequence fragmentation - Alternate screen exit detection (auto-clear after Claude Code no-flicker/vim) - SIGWINCH-based reattach (force redraw by toggling terminal size) - 429 error message with hint to increase MAX_CONCURRENT_SESSIONS - Replaced mlflow-tracing with mlflow-skinny 3.10.1 - PTY read chunk 4096→65536 bytes - Fixed repo name in deployment docs - Version bump to 0.17.0
dgokeeffe
added a commit
that referenced
this pull request
Aug 5, 2026
…nested wheels (#121) * fix(auth): configure Omnigent Databricks provider * feat(deploy): generic app.yaml via valueFrom resource references Replace the commented-out OMNIGENTS_* literals with active valueFrom resource references so app.yaml stays generic AND git-deploy-friendly (no per-deploy file edit). The host becomes resource-gated: ON when the omnigent-wheels (UC Volume) + omnigent-server-url (Secret) resources are attached, OFF otherwise (an unresolved valueFrom yields an empty string and omnigents_host_enabled() returns False). - OMNIGENTS_SERVER_URL -> valueFrom: omnigent-server-url (Secret) - OMNIGENTS_WHEEL_SPEC -> valueFrom: omnigent-wheels (UC Volume) - OMNIGENTS_FORCE_REINSTALL + ENABLE_SP_APIKEYHELPER now active (were commented out alongside the literals). Add attach_omnigent_resources.sh + make attach-omnigent-resources target to attach the two resources per app, idempotently, MERGING with existing resources (apps update --resources replaces, so merge avoids clobbering workshop challenge-repo-token). Run after grant_omnigent_host.sh. Makefile: GRANT_YAML var + OMNIGENT_SERVER_URL/WHEEL_VOLUME derivation + OMNIGENT_SECRET_SCOPE/KEY defaults for the new target. * fix(attach-resources): use SDK create_update with update_mask to avoid wiping git_repository apps update --json is a full-body write that CLEARS unset fields — it wiped the coda app's git_repository on a live git-linked app, breaking redeploys (learned during go-live validation). The CLI's create-update --json body shape is finicky (rejects 'resources' as an unknown field when update_mask is embedded). Switch the write step to the Apps SDK directly: create_update(app, update_mask='resources', app=App(resources=[...])) — a targeted field-mask patch that touches ONLY resources, preserving git_repository and other app fields. Validated against the live coda app (git_repository + both omnigent resources present after re-run). * feat(host-share): accept grant_user in /api/omnigent-host/share body The share endpoint previously granted only to the calling user (the owner), so you could self-share but not share to a teammate. Add an optional grant_user in the request body; when present, share to that user instead. The owner-gate is unchanged (only the app owner may invoke the endpoint, since it acts with the SP's authority). Default behavior (no grant_user) is unchanged: share to the calling user, so the browser auto-share path still works. * test: align fixtures with current setup behavior * chore: remove personal values from resource example * feat(host): resolve omnigent wheels from nested wheels/<version>/ dir _materialize_spec listed only the top level of the UC Volume, so it raised 'no .whl in UC Volume' when the omnigent build publishes wheels under wheels/<version>/ instead of flat at the volume root. Recurse (bounded depth) when the root has no wheels, pick the directory holding the newest-uploaded main omnigent- wheel, and install every wheel from that dir. Flat top-level layout still works unchanged. Tests cover flat, nested-newest-wins, and the no-wheels error. * fix(host): install jq so native-harness Databricks auth resolves Omnigent's native pi/claude/codex harnesses resolve their AI-Gateway bearer via an auth command ending in `... --output json | jq -r '.access_token'` (omnigent.inner.codex_executor._databricks_codex_auth_command). Databricks Apps containers ship no jq, so the pipe yields an EMPTY token and the harness fails with "Failed to resolve API key" — the reported pi-native breakage. Add install_jq.sh (static binary to ~/.local/bin, same fetch pattern as install_tmux.sh) and wire it into both paths: _run_step("jq") in app.py (interactive) and _ensure_jq() in omnigents_host.py before _run_setup_once (host). Idempotent; non-fatal on failure. * fix(proxy): strip JSON-Schema keywords the Gemini surface rejects opencode tool calls 400'd with "Unknown name exclusiveMinimum ... Cannot find field" on the mlflow chat-completions gateway surface, which validates tool schemas against an OpenAPI-3.0 subset (shared with the Gemini translator). exclusiveMinimum/Maximum are NUMBERS in JSON Schema draft-2020 but boolean modifiers in OpenAPI 3.0, so the parser rejects the whole request. Add exclusiveMinimum/Maximum, multipleOf, minLength/maxLength/pattern, minItems/maxItems/uniqueItems, and minProperties/maxProperties/patternProperties to GEMINI_UNSUPPORTED_SCHEMA_KEYS. strip_unsupported_schema_keys recurses so they're removed at any depth (incl. nested properties[N].value). These are advisory constraints — dropping them keeps the tool callable; Claude/GPT ignore them. Regression test added. * docs(plans): design note for adopting ucode on the interactive path Captures the ucode-duplication findings from the 2026-07-20 debug session: CoDA's setup_*.py reimplements ucode's per-agent config writers (the root of the exclusiveMinimum dialect bug), ucode's DATABRICKS_BEARER seam as the clean auth integration, and — critically — that ucode does NOT cover the Omnigent runner native-harness path (which is where the reported errors actually came from). Proposes migrating only the interactive path; keeps the shipped hotfixes. * chore(deploy): make aws-syd the committed app.yaml base The base app.yaml targeted an Azure workspace (adb-7405614666872455) with env-specific Otel/MLflow/challenge-repo config. The active deploy target is now the aws-syd workspace (coda app, dbc-d8b32eb0-866b, AWS ap-southeast-2): point DATABRICKS_GATEWAY_HOST there, GEMINI_MODEL=databricks-gemini-3-pro, ENABLE_SP_APIKEYHELPER=true, MAX_CONCURRENT_SESSIONS=5. Parity with the live deployed coda config (verified via workspace export). Retains app.yaml.aws-syd as the untracked source overlay. * fix(host): make omnigents-host profile SDK-authable for pi model catalog pi (and any omnigent native harness) showed only its single hard-coded default model (databricks-claude-sonnet-4-6) instead of the workspace's full endpoint list. Root cause: omnigent's pi_native_credentials resolves the model catalog via resolve_databricks_workspace("omnigents-host") → Config(profile="omnigents-host").authenticate(), but that profile is host-only (credentials are brokered over loopback, never persisted). The SDK raised "cannot configure default credentials", the catalog fetch failed, and pi fell back to one model. Two coordinated changes so the SDK can mint via the existing broker shim: - token_helper.write_databricks_token_wrapper: emit the FULL OAuth shape the SDK's DatabricksCliTokenSource requires (access_token + token_type + expiry), not a bare {access_token}. Broker gives no expiry and always mints fresh, so set a short now+5min expiry (well inside the ~1h SP TTL) → SDK re-invokes the shim for a fresh token rather than caching one of unknown lifetime. - omnigents_host._write_oauth_profile: write auth_type = databricks-cli so Config(profile=...).authenticate() runs `databricks auth token --profile omnigents-host`, which resolves to the broker shim (first on the runner PATH). Verified live: broker token → GET /api/2.0/serving-endpoints = HTTP 200, 50 endpoints (11 claude, 9 gemini, 19 gpt). Tests updated for the new shim shape + profile auth_type. * fix(host): route SDK CLI-auth through the broker shim by path, emit JSON always Follow-up to 017ab40 — that made the profile 'authable' in theory but two live failures on the AWS container (sdk 0.106.0) proved it insufficient: 1. The SDK's DatabricksCliTokenSource resolves the `databricks` binary via its OWN lookup, NOT $PATH — so it ran the REAL CLI in ~/.local/bin (no OAuth cache) → 'databricks OAuth is not configured'. Fix: write `databricks_cli_path = <broker shim>` into the omnigents-host profile. It's a normal Config attribute the SDK reads straight from .databrickscfg, so it reaches the in-runner catalog fetch WITHOUT needing omnigent's host→runner env allowlist (which we don't own / would need a wheel rebuild to change). 2. The SDK invokes `auth token --profile <p>` WITHOUT `--output json` yet json.loads()s stdout (verified in the container's 0.106.0 source). The shim only emitted JSON when --output json was present, so the no-flag path printed a raw token → 'cannot unmarshal CLI result: line 1 column 1'. Fix: shim now emits the access_token/token_type/expiry JSON unconditionally (matches the real CLI, which defaults auth token to JSON). Verified against the live container: broker token → serving-endpoints = 200, 50 models; shim --output json returns a valid JWT; DATABRICKS_CLI_PATH changes the SDK's error from 'OAuth not configured' to 'cannot unmarshal' (proving it now runs the shim). Tests updated.
dgokeeffe
added a commit
that referenced
this pull request
Aug 5, 2026
…ml files (#122) docs/agent-instructions.md §5 requires personal/workspace values to be commented out or defaulted off before landing on main. This is a public repo and several had accumulated: a customer workspace id, a UC catalog, a self-hosted MLflow app URL, and a personal sandbox repo. Each is both an information leak and actively wrong for anyone else deploying the template. Removed (commented out with placeholders, or already resolved via valueFrom): app.yaml DATABRICKS_GATEWAY_HOST adb-7405614666872455…azuredatabricks.net CLAUDE_CODE_OTEL_CATALOG_SCHEMA edp_aisandbox_aisandbox_dev.ppcs MLFLOW_OSS_URL coda-mlflow-oss-7405614666872455… CHALLENGE_REPO_URL github.com/david-okeeffe_data/dok-sandbox app.yaml.workshop the same four, plus OMNIGENTS_SERVER_URL / OMNIGENTS_WHEEL_SPEC switched to the valueFrom resources the base app.yaml already uses (#121) app.yaml.template DATABRICKS_GATEWAY_HOST — see below ## DATABRICKS_GATEWAY_HOST has three states, not two Unset is the right default and is why this one is commented rather than blanked: unset -> derive from the workspace id (or an Azure DATABRICKS_HOST) and PROBE for reachability (utils.get_gateway_host tier 2) set URL -> TRUSTED, no probe (tier 1) set "" -> explicitly DISABLE the gateway, use serving-endpoints So `value: ""` is not "no value" — it turns the gateway off. And because a set URL is trusted without probing, a stale one doesn't fall back: a token minted at one workspace is invalid at another's gateway, so every model call returns "400 Invalid Token". That last point is why the new guard caught a real bug in app.yaml.template, which shipped DATABRICKS_GATEWAY_HOST *uncommented* with the placeholder still in it. Deploying the template unedited would pin the gateway to the literal string `https://<your-gateway-id>.ai-gateway.<env>.cloud.databricks.com` and fail every model call, where leaving it out would have worked. Now commented, with the trap spelled out. ## Safety of each removal - CLAUDE_CODE_OTEL_CATALOG_SCHEMA: claude_otel.py returns False when it's absent, so OTEL is simply off. CLAUDE_CODE_OTEL_ENABLED also flipped to "false" so the flag doesn't advertise a feature that can't run. - MLFLOW_OSS_URL: proxy_tracing._enabled() already requires both the flag and a non-empty URL; the flag was already "false". - CHALLENGE_REPO_URL: app.py only registers the "challenge" setup step when it's set, and install_challenge_repo.sh exits 0 when unset. Commented together with CHALLENGE_REPO_READ_TOKEN, since an unresolved valueFrom for it is the source of the known "error resolving resource challenge-repo-token" boot warning. ## Guard tests/test_app_yaml_overlays.py grows two checks over every tracked app.yaml*: no active env value may contain a workspace id, a concrete UC Volume/catalog, an AWS workspace host or a personal GitHub repo; and DATABRICKS_GATEWAY_HOST must not be pinned. Commented-out examples are exempt — that's where placeholders belong. Verified the guard fails when the UC catalog value is reintroduced. 549 passed, 3 skipped. All four app.yaml* files still parse and still declare every ENABLE_<CLI> toggle.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings forward
feat/omnigent-host-followups(13 commits, 2026-07-20) — the last branch in the repo with unlanded work. It never had a PR. Rebuilt against currentmainrather than merged, for the reason in §1.1. Generic app.yaml — fixes a §5 violation on main
OMNIGENTS_SERVER_URLandOMNIGENTS_WHEEL_SPECbecomevalueFromresource references.docs/agent-instructions.md§5 says these must be commented out or defaulted off before landing on main; main currently carries one workspace's Omnigent server URL and UC Volume path.The on/off switch becomes "are the resources attached" rather than "did you edit this committed file" — an unresolved
valueFromyields an empty string andomnigents_host_enabled()returns False. Addsattach_omnigent_resources.sh+make attach-omnigent-resources, using SDKcreate_updatewith anupdate_maskso attaching doesn't wipegit_repository.CHALLENGE_REPO_URLstill holds a personal sandbox URL on main — left alone to keep this reviewable, worth the same treatment separately.2. Teammate host-share
POST /api/omnigent-host/shareaccepts an optionalgrant_userin the body, so the owner can share the SP-owned host with a teammate. Previously it could only self-share. This is exactly the gap §5 describes: "An SP-owned host is invisible in a human's personal picker unless shared viaPUT /v1/hosts/{id}/permissions/{user_id}". Stays owner-gated viabefore_request.3. SDK-authable broker shim
The CLI shim now emits the full
access_token/token_type/expiryJSON thatdatabricks-sdk'sDatabricksCliTokenSourcerequires — unconditionally, not only under--output json. The SDK builds the command without that flag yet stilljson.loads()the output (the real CLI defaultsauth tokento JSON), so the old raw-token path madeConfig(profile=...).authenticate()fail with "cannot unmarshal CLI result" and pi fell back to a single-model picker.Expiry is deliberately short (5 min) so the SDK re-invokes the shim for a fresh token rather than caching one whose real lifetime we can't prove. Verified no caller depends on the old raw output —
provision_coda_pats.shalready passes--output json, and nothing uses--output text.4. jq + nested wheel layouts
Omnigent's native harnesses end their auth command in
| jq -r .access_token; there's no jq in the Apps image, so without it the harness reports "Failed to resolve API key". Host wheels are now resolved fromwheels/<version>/subdirs, picking the dir with the newest-uploaded main wheel so a stale version alongside a new one isn't mixed in.Fixes made while integrating
setup_state["steps"]._update_step()silently no-ops on an unknown id, so the step would have run invisibly — no progress line, no error surfaced if it failed.install_jq.shignoredGITHUB_RELEASE_MIRROR, whichinstall_gh.sh,install_micro.shandinstall_databricks_cli.shall honour — a firewalled deploy couldn't reach github.com. Now mirrored; downloads to a temp path and verifies the binary runs before installing it asjq(a truncated download or HTML error page would otherwise land on PATH and silently produce an empty token — the exact failure this script exists to prevent); and is best-effort, warning and exiting 0 rather than marking the whole app setup errored, since jq only matters on the Omnigent path which is off by default.minLength/maxLength/pattern,minItems/maxItems,minProperties/maxProperties/patternProperties, with the clearer rationale that Gemini's OpenAPI-3.0 parser has no draft-2020 fields. Took the superset, kept both sets of tests.Verification
app.yamlenv var set diffed against main — only the two intended entries changedinstall_jq.shexercised with jq absent and a broken mirror: warns, exits 0, installs nothingvalueFromchange means Omnigent host attach is OFF on main until the resources are attached — which is the §5-intended state, but it is a behavioural change for anyone deploying main today.