Add package definitions, build script, and test suite#232
Open
peterjan wants to merge 2 commits into
Open
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces a new, Debian-policy-aligned packaging layout for Sia daemons by defining per-daemon package metadata/config under packages/<name>/, generating .deb artifacts via a shared scripts/build-deb.sh, and validating install/upgrade/migration/remove behavior with an end-to-end containerized test suite.
Changes:
- Add
scripts/build-deb.shand packaging templates (systemd unit, maintainer scripts, control, logrotate, copyright). - Add per-daemon package definitions and default configs for
hostd,renterd,walletd, ands3d. - Add a Docker-based packaging test suite + CI workflow to lint, build, and run package scenarios across Debian/Ubuntu variants.
Reviewed changes
Copilot reviewed 35 out of 36 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
.gitignore |
Ignore generated dist/ artifacts and .deb outputs. |
.github/workflows/test.yml |
Add CI workflow for shellcheck, Go checks, package build, and package scenario tests. |
scripts/build-deb.sh |
New build script that assembles .deb packages from templates + per-daemon definitions. |
scripts/templates/control.tmpl |
Package control file template. |
scripts/templates/copyright.tmpl |
Debian copyright file template. |
scripts/templates/logrotate.tmpl |
Logrotate config template for daemon logs. |
scripts/templates/postinst.tmpl |
Post-install maintainer script handling migration and restart policy. |
scripts/templates/postrm.tmpl |
Post-removal maintainer script handling purge and failure unwind. |
scripts/templates/preinst.tmpl |
Pre-install maintainer script handling legacy unit detection/stash. |
scripts/templates/prerm.tmpl |
Pre-removal maintainer script handling disable/stop policy. |
scripts/templates/service.tmpl |
Systemd unit template using Debian-expected locations and defaults. |
packages/hostd/package.env |
hostd package metadata. |
packages/hostd/hostd.yml |
Default hostd config shipped as a conffile under /etc. |
packages/renterd/package.env |
renterd package metadata. |
packages/renterd/renterd.yml |
Default renterd config shipped as a conffile under /etc. |
packages/s3d/package.env |
s3d package metadata. |
packages/s3d/s3d.yml |
Default s3d config shipped as a conffile under /etc. |
packages/walletd/package.env |
walletd package metadata. |
packages/walletd/walletd.yml |
Default walletd config shipped as a conffile under /etc. |
tests/README.md |
Documentation for running the packaging test suite locally. |
tests/Dockerfile.systemd |
Builds a systemd-enabled test image for running package scenarios. |
tests/lib.sh |
Container lifecycle helpers for the packaging tests. |
tests/run-tests.sh |
Orchestrates building the stub, building images, running scenario scripts. |
tests/container-prep.sh |
Container-side prep: copy sources, build debs, build legacy debs. |
tests/build-legacy-deb.sh |
Builds legacy-style .deb packages for migration testing. |
tests/lintian-suppressions.txt |
Lintian suppressions used by the lintian scenario. |
tests/stub/main.go |
Minimal stub daemon used by packaging tests. |
tests/scenarios/helpers.sh |
Scenario assertion helpers and systemd state helpers. |
tests/scenarios/01-fresh-install.sh |
Fresh install layout + conffile registration assertions. |
tests/scenarios/02-upgrade.sh |
Upgrade behavior assertions (restart-if-running, preserve enablement). |
tests/scenarios/03-config-preservation.sh |
Conffile + systemd drop-in preservation assertions. |
tests/scenarios/04-legacy-migration.sh |
Legacy migration assertions (stock vs modified unit behavior). |
tests/scenarios/05-remove-purge.sh |
Remove vs purge behavior assertions (keep config/data vs cleanup logs/config). |
tests/scenarios/06-lintian.sh |
Lintian check scenario (runs on Debian trixie only). |
tests/scenarios/07-failed-upgrade.sh |
Failed-upgrade unwind behavior assertions. |
tests/scenarios/nosystemd-install.sh |
Install/upgrade/remove/purge + legacy migration in no-systemd containers. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
peterjan
force-pushed
the
pj/deb-packages
branch
from
8ac483e to
0b9d6f0
Compare
| # builds a .deb package for a Sia daemon from a prebuilt linux binary and the | ||
| # package definition under packages/<project>. used by the build workflow and | ||
| # by the test suite so the tested artifact matches the shipped artifact. | ||
| set -euo pipefail |
Comment on lines
+86
to
+88
| # bash replacement keeps every substitution literal. the replacements must be | ||
| # quoted because bash 5.2 enables patsub_replacement by default, which gives | ||
| # unquoted & and backslash special meaning |
| @@ -0,0 +1,34 @@ | |||
| #!/usr/bin/env bash | |||
| # remove keeps configuration and data, purge removes configuration and logs | |||
| # but never wallet data | |||
Member
There was a problem hiding this comment.
Why leave wallet data? Seems a bit annoying to not be able to rely on purge to actually remove everything.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR lays the groundwork for closing out #129 and #179. I'll F/U with a PR that switches the build workflow over and closes both, if this gets approved of course. This PR restructures how our .deb packages are defined, without touching the release flow yet. It's quite lengthy because we migrate existing installs...
The main problem with our packages now as I understand it are:
I think the solution that tackles both of these problems is to ship a conffile.
A conffile is just a config file that the package manager treats as yours. It never overwrites it on upgrade. If we ever change the default and you changed yours too, apt shows you the diff and asks what to do instead of clobbering it. That is basically the exact thing #179 asked for, and shipping config under /etc the proper way is also what Debian wants, so it helps #129 at the same time.
So this PR adds a packages// folder per daemon with its metadata and a default config, and a build script that turns that into a .deb. Same script is used by CI and the tests, so what we test is what we ship. hostd, renterd, walletd and s3d are all in here. Adding a new one later is just one more folder.
What else the new packages do
Upgrading from the old packages
The first upgrade moves everything to the new layout for you. If you never touched the service file, we swap it for the new one. If you did edit it, we leave yours exactly as is. If it was set up and running, we turn it back on once after the upgrade. Old installs that already had a config keep using it, nothing changes for them.
There is a test suite that installs, upgrades, migrates and removes the packages inside real containers, across debian bullseye, bookworm, trixie and ubuntu jammy, questing. It is all green. CI runs it on every PR. Nothing here changes the actual release flow yet. The packages we publish stay the same until the follow up PR flips the switch.