chore(dev): tag dev images :dev-<sha> instead of :latest (no cross-worktree poisoning)#587
Merged
Merged
Conversation
OisinKyne
reviewed
Jun 3, 2026
OisinKyne
approved these changes
Jun 3, 2026
Contributor
OisinKyne
left a comment
There was a problem hiding this comment.
One worry that i'm not so sure the write will overwrite an existing file, i guess it probably does?
7b9c479 to
36146d0
Compare
Under OBOL_DEVELOPMENT, both the manifest rewrite (internal/defaults) and the image build (internal/stack) used a shared, mutable `<image>:latest` tag. On a host running more than one obol-stack worktree against the same Docker daemon, one worktree's build silently overwrites :latest and a different worktree's `obol stack up` then deploys the wrong binary — observed during rc9 QA, where a sibling branch's serviceoffer-controller (reading a per-purchase Secret this branch's RBAC doesn't grant) poisoned an unrelated stack and presented as a hang, not an obvious image mismatch. Tag dev images with `dev-<short-git-sha>` instead: - defaults.DevImageTag() = dev-<sha> of the working tree (`latest` fallback when not a git checkout, preserving prior behaviour for tarball builds). - CopyInfrastructure stamps the rendered manifests with the dev tag AND persists it to $CONFIG_DIR/.dev-image-tag, so internal/stack builds/imports the exact tag the cluster pins — even if HEAD moves between `stack init` and `stack up`. - Different branches/worktrees get distinct tags (no cross-worktree poisoning); committing changes the SHA and triggers a fresh build; uncommitted changes reuse the committed tag unless OBOL_FORCE_REBUILD_LOCAL_DEV_IMAGES is set. Validated: go test ./... (33 pkgs); `obol stack init` (dev) rewrites both the controller and buyer pins to :dev-<sha> with no leftover @sha256 and persists the tag; `docker build -t …:dev-<sha>` + `k3d image import` succeed (the exact build/import path buildAndImportLocalImages runs).
36146d0 to
18617a6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
What changed: Under
OBOL_DEVELOPMENT, locally-built images are now tagged<image>:dev-<short-git-sha>instead of the shared, mutable<image>:latest. Themanifest rewrite (
internal/defaults) and the image build (internal/stack) agree onthe tag via a persisted
$CONFIG_DIR/.dev-image-tag.Why it matters:
:latestis shared across every obol-stack worktree on one Dockerdaemon. One worktree's build silently overwrites
:latest, and another worktree'sobol stack upthen deploys the wrong binary. This bit rc9 QA: a sibling branch'sserviceoffer-controller(which reads a per-purchase Secret this branch's RBAC doesn'tgrant) poisoned an unrelated stack and presented as a buy hang, not an obvious image
mismatch. A per-commit
dev-<sha>tag isolates builds by source so this can't recur.Bonus: committing changes the SHA and auto-triggers a fresh build.
Risk level: low — dev-only path (production digest pins untouched);
:latestfallbackpreserves behaviour for non-git/tarball builds; the build loop is otherwise unchanged
(only the tag string differs).
Commit under test:
7b9c479dBase branch:
mainScope
Validation
Unit tests:
Manual / runtime:
Release smoke: not re-run for this change (dev-build tagging only). The build/import
path it touches is the same one release-smoke exercises every run; the only behavioural
delta is the tag string, validated above. A full dev
obol stack upis the recommendedfinal confirmation.
Review Notes
Known gaps:
internal/defaults.devLocallyBuiltImageBasesandinternal/stack.baseLocalImagesremainduplicated (intentional, to avoid a defaults→stack import cycle) — unchanged by this PR.
dev-<sha>(same caveat as the old:latest);OBOL_FORCE_REBUILD_LOCAL_DEV_IMAGESis the escape hatch.Reviewer focus:
internal/defaults/defaults.go:DevImageTag/ReadDevImageTag, the persist inCopyInfrastructure, andrewriteDevDigestPins(defaultsDir, devTag).internal/stack/stack.go:buildAndImportLocalImagesreads the persisted tag and builds/imports
<base>:<devTag>(force-rebuild short-name matching still works vialocalImageShortName).