Conversation
The Transaction.State.Fail event is the only transaction event emitted
from a fire-and-forget ioScope.launchWithTracking{} coroutine (Start,
Abandon and Pending track inline within the awaited purchase() chain).
The test_purchase_failed_* tests sampled events.value after
advanceUntilIdle(), racing that detached emission and intermittently
missing the Fail event. Await it via events.first { predicate },
mirroring ObserverModeTest's deterministic flow-await idiom.
…test Stabilize TransactionManagerTest timing and threading
Add system font size as a device attribute
Fix paywall instances overwriting each other's experiment id
On devices without the Play Store, the billing client reports BILLING_UNAVAILABLE and product fetches throw BillingNotAvailable, which failed the whole paywall request even when test mode was active and products should have come from the test catalog instead. Two gaps caused this: - Race on activation: the test product catalog is fetched over the network when test mode activates, but a paywall requested right after config landed read an empty catalog and fell through to Play billing. StoreManager now waits (up to 5s) for the catalog to load before resolving products while test mode is active. - No test-mode exemption: any product id not resolved from the test catalog hit Play billing, and a BillingNotAvailable error failed the entire load, discarding the test products that did resolve. In test mode, BillingNotAvailable is now swallowed and whatever resolved from the test catalog and caches is served. Also stop re-evaluations from replacing an active test-mode session, which wiped the loaded catalog and entitlement selections and left the new session's products forever unloaded. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0117xhsmLu2YmArN8632vmtF
…egration test When test mode re-activates for a different reason (e.g. another test user matched after identify), the simulated entitlements, selections and overridden subscription status are now reset while the user-independent product catalog is kept. Also add a JVM integration test that wires the real GoogleBillingWrapper to a BillingClient whose setup finishes with BILLING_UNAVAILABLE — the exact signal a device without the Play Store produces — and verifies the StoreManager + TestMode chain serves test-catalog products (including after the billing failure is permanently cached), while non-test-mode loads still fail as before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0117xhsmLu2YmArN8632vmtF
Adds a CI benchmark that measures the time from preloadAllPaywalls() until every paywall for the dev app's embedded API key reaches PaywallLoadingState.Ready. - PaywallPreloadBenchmark instrumented test in :app runs N iterations (cold + warm), polls the SDK view store for Ready state, and writes JSON results to the device's Downloads collection - preload-benchmark.yml workflow runs on PRs into main (release PRs) across 3 parallel emulators (LOW/MID/HIGH tiers) pinned to the same Play Store device image (pixel_6, API 34, google_apis_playstore), differing only in cores/RAM, with a 5s settle after boot - scripts/benchmark_compare.py aggregates tier results, writes the report to .benchmark/, and fails the run when a tier's mean regresses more than the delta limit (default 10%) over its committed baseline - Baselines are recorded via workflow_dispatch with update_baseline; each result records the SDK's own DeviceClassifier tier so the emulator configs can be verified against real tier classification Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DiCiN2qg1dCDh7TqJMYTY1
…ting workflows - Also trigger on push to main/develop so merged branches keep fresh benchmark reports - Opt in to ExperimentalCoroutinesApi for resetReplayCache (verified :app:compileDebugAndroidTestKotlin passes) - Document that the emulator setup mirrors pr-tests.yml/build+test.yml and that the pixel_6 Play Store device matches maestro-run.yml's DeviceCloud config Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DiCiN2qg1dCDh7TqJMYTY1
Each emulator now invokes connectedDebugAndroidTest runsPerEmulator times (default 3); every invocation reinstalls the app and clears its data, so each run's first iteration is a true cold start. The test tags results with a run index and writes one JSON per run, and the compare script merges all runs per tier, computing the gated mean (plus cold/warm means, median, stddev, CV) across all runs x iterations. The AVD snapshot prewarm stays separate from pr-tests.yml on purpose: those AVDs are API 33 google_apis pixel_7_pro without Play Store, while the benchmark needs the API 34 google_apis_playstore pixel_6 image, and per-tier cores/RAM are baked into each snapshot. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DiCiN2qg1dCDh7TqJMYTY1
…d benchmark on develop PRs test_preload_active_paywall in :superwall androidTest still asserted the pre-#430 behavior where updatePaywall mutated the cached paywall, leaking presentationSourceType onto the instance returned by the preload-of-active-paywall path. Since ad31f3a returns a copy, the cached paywall no longer carries the presenter's source type, so the test now verifies the presented copy kept its request-specific values and that preloading didn't clobber the original experiment (mirroring the unit test updated in #430). Verified by replicating the scenario as a JVM test: old assertion fails on presentationSourceType, new assertions pass. The benchmark workflow now also runs on PRs into develop when the PR carries the 'benchmark' label (release PRs into main still always run), so the pipeline can be exercised before merge. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DiCiN2qg1dCDh7TqJMYTY1
…te shells android-emulator-runner passes each line of its script input to its own sh -c, so the multi-line for-loop died with 'Syntax error: end of file unexpected' on all three tiers. Move the settle/loop/pull sequence into scripts/run_preload_benchmark.sh and invoke it as a single line. Verified with bash -n and a stubbed adb/gradlew dry run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DiCiN2qg1dCDh7TqJMYTY1
…per run The gradle build previously ran inside the emulator session, competing with the emulator for the runner's cores; assembling :app debug + androidTest APKs beforehand means the three in-session connectedDebugAndroidTest invocations only install and run (the class filter already restricts execution to the benchmark test). Iterations per run drop from 3 to 2 — still 3 fully cold runs per emulator, averaged over 6 samples. The Play Store system image download and AVD snapshot creation were first-run-only costs; subsequent runs restore the per-tier cached AVD. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DiCiN2qg1dCDh7TqJMYTY1
… am instrument A single build-apks job assembles both APKs and shares them as an artifact; the three tier jobs download them, adb install once, and per run do pm clear + am instrument directly. This removes the JDK/gradle setup and the ~2-3 min of gradle configuration+install overhead from each of the 3 in-session runs while keeping the cold-start guarantee (pm clear wipes app data before every run). am instrument exits 0 through adb even on failure, so the script parses the instrumentation output for OK/FAILURES/crash markers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DiCiN2qg1dCDh7TqJMYTY1
All three benchmark tiers hung for 90+ minutes because the wait for ConfigurationStatus.Configured sat outside the iteration timeout: when configuration fails or stalls on the emulator the status never becomes Configured and the test blocked forever. The wait is now bounded (benchmarkConfigureTimeoutSec, default 120s), fails fast with the actual status on ConfigurationStatus.Failed, and the tier jobs carry timeout-minutes: 30 so a hang can never ride to GitHub's 6h default. On a failed run the script now dumps filtered logcat so the root cause is visible in the CI log. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DiCiN2qg1dCDh7TqJMYTY1
… runners tee failed with 'No such device or address' in the runner's sh -c context; combined with set -e + pipefail this killed the script after am instrument completed and swallowed its output, so all three tiers failed without diagnostics. Capture plain, print after, tolerate adb's exit status, and log per-run wall time. Result upload downgraded to if-no-files-found: warn so failed runs don't stack a misleading artifact error on top (the report job already fails on missing tiers). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DiCiN2qg1dCDh7TqJMYTY1
All three tiers failed fast with 'SDK configuration timed out (status still Pending)': GitHub runners resolve via systemd-resolved on 127.0.0.53, which the emulator guest cannot use, so api.superwall.me never resolves and the config fetch retries forever. Nothing else in CI does real network from a GH-hosted emulator (superwall tests mock it; real-network app tests run on DeviceCloud), which is why this never surfaced. Pass -dns-server 8.8.8.8 to both emulator steps, add a guest/host connectivity pre-flight to the script, and dump an unfiltered logcat tail on failure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DiCiN2qg1dCDh7TqJMYTY1
…s a network DNS now resolves in the guest (the -dns-server fix worked: the pre-flight printed api.superwall.me's IP, and the host reaches the API with HTTP 200), yet configuration still timed out at Pending. The SDK's awaitUntilNetworkExists() requires an active network with a WIFI or CELLULAR transport, and netsim WiFi does not reliably come back after the AVD snapshot restore — the guest believes it is offline even though the NAT path works, so the config fetch never starts. Toggle WiFi off/on after boot, wait bounded for an active default network, and print the connectivity state in the pre-flight. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DiCiN2qg1dCDh7TqJMYTY1
…rors ConnectivityManager now reports a validated LTE network 2s after the WiFi cycle and DNS resolves in the guest, yet configuration still never leaves Pending — the config request is failing and silently retrying (getConfig's retry callback flips state to Retrying, which maps to Pending externally, and nothing logs at error level until retries are exhausted). Run the benchmark with LogLevel.debug and widen the failure logcat window (+System.err, since HandleFetchFailure uses printStackTrace) so the next run shows the actual exception. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DiCiN2qg1dCDh7TqJMYTY1
Adds present_video, present_localized, present_free_trial, present_data_v4, scroll_test and price_readout to the existing four — covering video, localized, free-trial, long-scroll and price-heavy paywalls alongside the product-less and product-bearing ones. Known traps stay excluded: no_paywalljs never reaches Ready by design, holdout placements resolve to no paywall, incorrect_product_identifier is deliberately broken. The SDK dedupes placements sharing a paywall, so the report's paywall count is the source of truth for set size. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DiCiN2qg1dCDh7TqJMYTY1
Ten new placements (surveys, state/style/theme tests, subscription variants, consumable, non-recurring) join the existing ten. Ten cold runs per tier give the statistics room to settle at this workload size; per-iteration timeout rises to 240s and the tier job cap to 40 minutes to accommodate the larger set on the low tier. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DiCiN2qg1dCDh7TqJMYTY1
At 20 placements (~25 paywalls) the concurrent WebView load saturated every tier: HIGH measured slower than MID, CVs hit 39-49%, and runner noise drowned the hardware signal the delta gate exists to protect. The 10-paywall set kept tier ordering monotonic with markedly lower variance. Keeps 10 cold runs per tier and the enlarged timeouts as headroom. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DiCiN2qg1dCDh7TqJMYTY1
Deltas derived from measured spread of the 10x10 configuration (CV: mid 9.5%, high 17.6%, low 38.3% -> ~3-sigma on run-to-run mean comparisons): low 30%, mid 15%, high 20%, default 20%. The gated metric switches to medianMs since low/high means are dragged by outlier runs the median shrugs off. Baselines now update automatically on every push to main/develop, so release PRs always compare against the latest mainline numbers; manual update_baseline dispatch remains. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DiCiN2qg1dCDh7TqJMYTY1
…down Fix background teardown
Feature PRs into develop no longer run the benchmark (the label-gated opt-in is removed); release PRs into main, merges to main/develop (which roll baselines forward) and manual dispatch remain. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DiCiN2qg1dCDh7TqJMYTY1
…rk-ci-b56ahw Add paywall preload benchmark CI for release PRs
Building the device template assembled ~55 fields and ran a full encodeToString/decodeFromString round trip on every call — per rule, per evaluation, plus twice concurrently during config fetch. Each build also repeated system IPCs (Settings.Secure ANDROID_ID, PackageManager lookups) and allocated six SimpleDateFormats. - Cache the serialized template keyed by a fingerprint of every mutable input (identity, entitlements, subscription status, active products, interface style, platform wrapper, locale/timezone, storage counters, storefront). Any change to those inputs rebuilds the template; enrichment and experimental properties stay outside the cache and are applied fresh. - Overlay time-derived fields (utc/local date-time strings, days/minutes since install and last paywall view) on every read instead of keying the cache on them, deriving all of them from a single timestamp. - Memoize process-constant fields: vendorId/deviceId (Settings.Secure), app version/build/install date (reuse the constructor-cached PackageInfo), padded versions, capabilities JSON, kotlinVersion. - Replace the JSON string round trip in DeviceTemplate.toDictionary with encodeToJsonElement and a direct JsonObject-to-map conversion. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ukxof6pYoqDdcFMvwCtb23
8615b5d wrapped message parsing in runCatching so unknown messages don't crash the batch, but left the original unguarded add in place. Every message was added twice, and an unknown message still threw and failed the whole parse. Keep only the guarded add and log a warning when a message can't be parsed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ukxof6pYoqDdcFMvwCtb23
Follow-ups to get CI green after the develop changes this branch sits on: - parseWrappedPaywallMessages: skip messages with unknown event names (logging a warning) instead of catching every parse error. Newer paywalls can send events an older SDK doesn't know about without breaking the batch, while a malformed payload for a known event still fails the whole parse, as parsePageView_missingRequiredField_fails expects. Update parse_unknown_event_returns_failure to the new semantics and have it prove a known message alongside the unknown one still parses. - PaywallViewDismissTest: pass forceCleanup = true to beforeOnDestroy/ destroyed. Since 4a4ad72 those are no-ops on non-finishing stops; production reaches teardown via forceCleanup = isFinishing, and these tests simulate a dismissal that finishes the Activity. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ukxof6pYoqDdcFMvwCtb23
Feature PRs land in develop, so preload regressions were only caught at release time when a PR into main ran the benchmark. Trigger it on PRs into develop too; merges already kept baselines fresh via push. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ukxof6pYoqDdcFMvwCtb23
…ion-cjke25 Memoize device template to improve rule evaluation performance
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes in this pull request
Checklist
CHANGELOG.mdfor any breaking changes, enhancements, or bug fixes.ktlintin the main directory and fixed any issues.