backup: add gc command - #1341
Merged
Merged
Conversation
bigbes
force-pushed
the
bigbes/tntp-8671-backup-gc
branch
from
July 31, 2026 07:02
e099f16 to
674d280
Compare
bigbes
force-pushed
the
bigbes/tntp-8671-backup-gc
branch
from
July 31, 2026 07:48
674d280 to
b009611
Compare
sssciel
approved these changes
Jul 31, 2026
bigbes
force-pushed
the
bigbes/tntp-8671-backup-gc
branch
from
July 31, 2026 08:52
b009611 to
9c92a63
Compare
Add `tt backup gc`, the retention-based cleanup of a backup storage. It builds a deletion plan out of the manifest chain and only then executes it, so `--dry-run` shows what a real run would remove. Retention combines `--keep-full N` (the last N healthy chains) and `--keep-days D`: a backup goes only when no rule keeps it, and without either flag nothing is deleted at all. Deletion cascades and cuts a chain from its newest end only, so a kept increment always keeps the backups it is recovered from; a full backup goes only with its whole chain. Inside one backup the manifest goes before its archives, and inside a chain the newest backup goes first, so an interrupted run leaves a valid chain prefix and collectable garbage rather than manifests pointing at nothing. Before anything is deleted the plan is filtered once more against every manifest that stays behind, since a manifest may name a previous backup in another chain or an archive another backup uploaded, and chain.Build treats neither as a problem. Three things are never deleted by the retention rules. The chain holding the newest manifest, because an upload may be appending an increment to it. The newest chain that can still be recovered from, so that a storage whose newest manifest belongs to a tail without its full backup does not end up losing every usable backup and keeping the unusable one. And an archive whose backup id is newer than every stored manifest, or any archive at all when no manifest is stored yet, since archives are uploaded before the manifest that references them. All three are stated in the output, together with the consequence: gc cannot empty a storage. Dangling archives are removed once older than `--orphan-age` (24h by default, an upload has to outlive it), each after a direct GET of its manifest, because LIST is not read-after-write consistent. Chains with problems are left to `tt backup verify` to report, except a tail whose full backup is gone and whose every manifest is older than `--orphan-age`: nothing can be appending to it and nothing else would ever collect it, so it is collected even when it holds the newest manifest. An archive is also left alone when the manifest of its backup is structurally invalid. Only the manifest says which archives a backup refers to, so one that cannot be believed makes its archives merely unclassifiable, not abandoned - and gc is the one command whose mistakes cannot be undone. One supporting change falls out of this: `ClusterManifest.Validate` now requires `creation_time`, which the schema has always marked mandatory. Every retention rule reads it, and a missing one is either "ancient" or "immortal" depending on the rule. Closes TNTP-8671
bigbes
force-pushed
the
bigbes/tntp-8671-backup-gc
branch
from
July 31, 2026 10:25
9c92a63 to
9afa0b3
Compare
patapenka-alexey
approved these changes
Jul 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add
tt backup gc, the retention-based cleanup of a backup storage. It builds a deletion plan out of the manifest chain and only then executes it, so--dry-runshows exactly what a real run would remove.Retention combines
--keep-full N(the last N healthy chains) and--keep-days D: a backup goes only when no rule keeps it, and without either flag nothing is deleted at all. Deletion cascades and cuts a chain from its newest end only, so a kept increment always keeps the backups it is recovered from, and a full backup goes only with its whole chain. Inside one backup the manifest goes before its archives, and inside a chain the newest backup goes first, so an interrupted run leaves a valid chain prefix and collectable garbage rather than manifests pointing at nothing. Before anything is deleted the plan is filtered once more against every manifest that stays behind, since a manifest may name a previous backup in another chain or an archive another backup uploaded, andchain.Buildtreats neither as a problem.Four things are never deleted, whatever the rules say. The chain holding the newest manifest, because an upload may be appending an increment to it. The newest chain that can still be recovered from, so a storage whose newest manifest belongs to a tail without its full backup does not lose every usable backup and keep the unusable one. An archive whose backup id is newer than every stored manifest — or any archive at all when no manifest is stored yet — since archives are uploaded before the manifest that references them. And the archives of a structurally invalid manifest: only the manifest says which archives a backup refers to, so one that cannot be believed makes its archives unclassifiable rather than abandoned. All of this is stated in the output, together with the consequence: gc cannot empty a storage.
Dangling archives are removed once older than
--orphan-age(24h by default, an upload has to outlive it), each after a direct GET of its manifest, because LIST is not read-after-write consistent.Chains with problems are left to
tt backup verifyto report, except a tail whose full backup is gone and whose every manifest is older than--orphan-age: nothing can be appending to it and nothing else would ever collect it, so it is collected even when it holds the newest manifest.One supporting change falls out of this:
ClusterManifest.Validatenow requirescreation_time, which the schema has always marked mandatory. Every retention rule reads it, and a missing one is either "ancient" or "immortal" depending on the rule.Stacked on #1338.