Ergonomics round 2 - #7
Merged
Merged
Conversation
The v8.1.2.5 release run died in the net10 emulator leg with a boot timeout.
The real cause was several minutes earlier in the log:
FATAL | Not enough space to create userdata partition.
Available: 5666.53 MB, need 7372.80 MB.
By the time the AVD is created the job has installed three .NET SDK bands,
three Android workloads, three platform SDKs, a system image and the packed
packages, and the runner has no room left. The net8 leg on the same commit
passed 14/14, so nothing was wrong with the code - the legs just landed on
different sides of the disk budget.
Reclaims ~10 GB of things this job never uses, the NDK first: no native code
is compiled here (the .aar ships prebuilt .so files), so it is ~5 GB of dead
weight sitting on the same volume as the AVD. Leaves /usr/share/dotnet and
the rest of the Android SDK alone, and prints df before and after so the next
disk regression is visible at a glance instead of surfacing as a boot
timeout.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
.NET for Android never feeds an .aar's embedded proguard.txt to R8 - _CalculateProguardConfigurationFiles assembles only the SDK defaults, the generated Xamarin config and @(ProguardConfiguration) - so upstream's own consumer rules were silently lost, and the two hand-written keep files (WebView, Ndk) covered two of thirteen JNI-only entry surfaces. build/generate-r8-rules.sh now writes buildTransitive/<PackageId>.pro for every row in packages.tsv: upstream's consumer rules recovered verbatim from the .aar (downloaded from Central and verified against the checksum pins before anything is read out of it) plus a curated keep list for each module's JNI-reached entry surface - the classes C# constructs, the interfaces whose members C# invokes, and the enums whose constants C# reads. Curated names without wildcards are checked to exist in the .aar. The pairing .targets files attach the rules only where R8 runs, and the pack wiring moves into Datadog.Binding.props so a package cannot opt out by forgetting the block - deliberately without Exists(), so a new package whose rules were never generated fails its pack instead of shipping bare. The package tests now assert the pair in all thirteen packages. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The device tests forced Java shrinking off unconditionally, so nothing ever ran the packages' keep-rules - a stale or missing keep would have shipped and surfaced as a consumer's Release-only ClassNotFoundException. The 'no shrinking' block is now a default rather than a rule: -p:DatadogDeviceTestsLinkTool=r8 turns R8 on, run-emulator-tests.sh grew a third argument to pass it through, and the e2e matrix adds one shrunk leg (net10 only - the rules ship identically in every target framework, so one leg is enough). The module-loadability smoke check then fails functionally on any class R8 strips. The pack job also regenerates the .pro files with generate-r8-rules.sh and fails on drift, and the README's shrinking guidance now describes rules shipped by all thirteen packages instead of two. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
VerifyDatadogMavenArtifactHashes checked only files whose names appear in the pin list - the _DatadogNotPinned set waves through the .aars that arrive inside NuGet packages - so a declared @(DatadogMavenArtifact) whose pin was never generated sailed past looking exactly like one of those, and the only backstop was the blunt 'nothing matched at all' error. Expected file names are deterministic (%(ArtifactId)-%(Version).%(Packaging)), so the target now requires every declared artifact's expected name to be present in build/maven-checksums.txt and errors with the missing names and a pointer to UpdateMavenChecksums.sh. CI additionally regenerates the pin list and fails on drift, so a hand-edited pin or a skipped regeneration is a named diff rather than a silent divergence. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The header claimed 'upstream signs nothing this build can check offline' and 'Google's Maven does not serve .sha256' - both wrong. Central serves a detached .asc for every com.datadoghq file, signed by Datadog's dd-sdk-android packaging key, and dl.google.com does serve .sha256 sidecars (the code already used them; only the comment said otherwise). When gpg is available the script now downloads each Datadog artifact plus its .asc and verifies the signature against the release signing key, pinned by FULL fingerprint (CAF18D4EC00CA4450C6725A59333D4EF32A49F0A, uid 'Datadog dd-sdk-android Packaging') - derived from the .asc's issuer key id and keyserver.ubuntu.com, with the derivation recipe recorded next to the constant. The keyring holds exactly that key, so a signature by any other key cannot verify, and the VALIDSIG status line is matched against the pinned fingerprint as well. Where the bytes were downloaded anyway, the published .sha256 must agree with them. Missing gpg is a loud skip warning; a failed fetch of the key with gpg present, a bad signature, or an inconsistent sidecar is a hard failure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The NOT BOUND section claimed Datadog publishes 27 dd-sdk-android* artifacts; Central's com/datadoghq index has 41 (counted 2026-07), and twelve of them were not accounted for at all. Every entry now carries a reasoned line: the -gradle-plugin-kcp-* family is Gradle build-time tooling and not bindable, -cronet and -apollo join the third-party instrumentation group, -coil3 joins its Coil sibling, -rum-debug-widget is Compose developer tooling with no MAUI meaning, and -flags, -flags-openfeature and -profiling are recorded as binding candidates that are not yet bound. Entries are now FULL artifact ids, one per line, because the section doubles as the machine-read roster: the upstream-watch workflow greps the ids out of this file and opens an issue when Central grows an artifact that is neither bound nor listed. The grep-derived roster is verified to equal Central's current listing exactly, and every existing consumer of the file (BuildNugets.sh, both workflows, run-emulator-tests.sh, the package tests) skips comment lines unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Upstream publishes a -sources.jar for every bound module, and the binding generator imports documentation from @(JavaSourceJar) into the generated C# XML docs - but nothing wired it, so GenerateDocumentationFile shipped undocumented members and IntelliSense showed signatures only. DownloadDatadogJavaSourceJars now fetches the bound artifact's sources jar on every target framework - @(AndroidMavenLibrary) never resolves sources, so this reuses the DownloadFile mechanism on all bands - and feeds it to the generator via @(JavaSourceJar), scheduled before the SDK's javadoc extraction targets evaluate their conditions. The jar is pinned in build/maven-checksums.txt and hash-verified like every other download: UpdateMavenChecksums.sh now emits a Bind column and pins (and, with gpg, PGP-verifies) the -sources.jar of every Bind=true artifact, the completeness guard expects those names, and the sorts run under LC_ALL=C so the committed order cannot depend on a machine's collation. Expectations stay modest - java-source-utils reads Java sources and dd-sdk-android is Kotlin, so KDoc yields less than a Java library would and CS1591 stays suppressed. All twelve sources jars' PGP signatures were verified against the pinned Datadog key before their hashes were recorded. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- README gains the NU1107 troubleshooting entry the device tests already documented from the inside: every net8 consumer of these packages needs a direct Xamarin.AndroidX.SavedState 1.4.0 reference to settle the Navigation.Common vs SavedState.Ktx disagreement, scoped to net8 because the newer bands settle higher on their own. - The SessionReplayCompose project claimed its PackageReferences 'have to be re-derived by hand when upgrading' - stale since verify-transitive-deps.py started re-checking them against the module's Gradle metadata on every CI run. - CheckReadmeVersions.sh matched run-simulator-tests.sh, an iOS-repo leftover this README never contains; the pattern is emulator-only now. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A weekly workflow (plus workflow_dispatch) compares two things against Maven Central: DatadogNativeVersion against dd-sdk-android-core's maven-metadata.xml <latest>, and the com/datadoghq artifact listing against the roster greppable out of build/packages.tsv - the bound rows plus the full ids in the NOT BOUND block, which is what made that block machine-readable. Findings become issues, idempotently: the title carries the finding (version, or the sorted list of unknown artifacts), open titles are compared exactly rather than through fuzzy search, and an open issue mutes re-filing until the finding changes shape. Both halves were dry-run against live Central: 41 artifacts, zero unaccounted, latest equals bound. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The tag drives which dd-sdk-android line is bound, and the checkout is the tagged commit - so a v3.14.0.1 tag on a commit whose Directory.Build.props still pins 3.12.1 would publish packages whose version number lies about what the commit's pins, keep-rules and README were maintained against. The version job now fails on that mismatch. Releasing an older line stays a non-event, because tagging the commit that built it means the props agree by construction; deliberately releasing a mismatched line remains possible through the documented DATADOG_ALLOW_NATIVE_MISMATCH repository variable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The convenience-API callout said the sources jars 'cannot currently be otherwise' imported; DownloadDatadogJavaSourceJars changed that. Say what is true now: the -sources.jar is fed to the generator and pinned like every other artifact, the importer parses Java sources, and Kotlin-heavy jars therefore yield little - the practical guidance stands. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
build/BumpNativeVersion.sh <version> does, in order and stopping at the first failure: DatadogNativeVersion := <version> and binding revision := 1 in Directory.Build.props, UpdateMavenChecksums.sh (pins before rules, because generate-r8-rules.sh refuses an unpinned .aar), generate-r8-rules.sh, verify-transitive-deps.py, the README's pinned versions (the checked snippets plus the badge and diagram - the version-scheme prose stays), CheckReadmeVersions.sh as proof, and a docs/release-notes/<version>.md scaffold. It ends by printing the steps that are judgement rather than mechanics. The README's upgrade section now leads with the script and keeps only the judgement steps and the .module-not-.pom spelunking guidance. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Binding revision 3 -> 4; the native SDK stays 3.12.1. Curated release notes lead with what consumers feel: complete R8 keep-rules across all thirteen packages (upstream's recovered consumer rules plus the curated JNI entry surface, exercised by a shrunk emulator leg), IntelliSense from upstream's KDoc, and the PGP-verified supply chain behind the 30-pin checksum list. README snippets move to 3.12.1.4; CheckReadmeVersions.sh is green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The module-loadability check proves each .aar reached the app by resolving marker classes by name. Under R8 that is the right assertion for every Datadog entry point - surviving the shrink is what the packages' keep-rules are for, and what the r8 leg exists to prove - but the two embedded third-party libraries (Kronos, JCTools) are asserted through exact classes no keep-rule covers, and a shrinker legitimately drops whatever of an embedded library the reachable SDK code does not use. The r8 leg would have failed on a legitimate removal. DATADOG_SHRUNK, defined whenever DatadogDeviceTestsLinkTool is set, excludes just those two markers; the unshrunk legs keep asserting them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The fallback description was imprecise: on a band without the javadoc anchors the download would still run through the hash verification's DependsOnTargets - the jar downloads and verifies, and simply feeds no docs. Say that, rather than 'nothing else changes'. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The pin-completeness drift step runs UpdateMavenChecksums.sh on every pull request, and with gpg present the script hard-failed when keyserver.ubuntu.com was unreachable - a third party's availability deciding whether unrelated PRs pass. The key material now lives in the repository (fingerprint verified against the pinned CAF18D4EC00CA4450C6725A59333D4EF32A49F0A before being committed, by parsing the armored packet per RFC 4880); import-from-file replaces the fetch, the keyserver stays as fallback for a checkout that lost the file, and either path ends in the same fingerprint check - the pin, not the transport, was always the trust anchor. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.