Skip to content

fix(metrics): harden attribute snapshots, retry backoff, and config validation#742

Draft
DanielVisca wants to merge 1 commit into
mainfrom
posthog-code/metrics-review-followups
Draft

fix(metrics): harden attribute snapshots, retry backoff, and config validation#742
DanielVisca wants to merge 1 commit into
mainfrom
posthog-code/metrics-review-followups

Conversation

@DanielVisca

Copy link
Copy Markdown
Contributor

💡 Motivation and Context

Follow-ups from the #739 review — approved to unblock, but three considerations went unaddressed before merge:

  1. Nested attribute values weren't snapshotted_SeriesState shallow-copied the attributes dict, so mutating a nested list/dict after count() rewrote an already-keyed series' attributes on the wire. Attributes are now deep-copied at capture (with a shallow fallback for un-copyable exotic values, keeping the no-throw guarantee).

  2. Retry policy was fixed-cadence with an off-by-one drop — the window was retried at the base flush interval and dropped on the fourth failure while the release note said three. Failed flushes now back off exponentially (2× per consecutive failure, capped at 64× the flush interval — the same ceiling the shared JS logs implementation uses) and the window is dropped after exactly _MAX_CONSECUTIVE_SEND_FAILURES = 8 consecutive failures (~21 min of outage coverage at the default 10s interval). The series cap still bounds buffered memory throughout. Browser/Node parity for this policy is a follow-up on the JS side.

  3. Lazy metrics init escaped the client's no-throw contractClient("phc_...", metrics={"resource_attributes": "bad"}) made the first client.metrics.count() raise AttributeError into application code. Nested config is now validated field-by-field (non-dict config/resource_attributes, non-numeric flush_interval, non-integer max_series_per_flush, non-callable before_send) and degrades to defaults with a warning; the client.metrics property additionally falls back to a default-config instance if construction fails (raising only in debug mode, matching no_throw).

💚 How did you test it?

TDD — new tests written first and confirmed failing against the merged code, then made green:

  • test_nested_attribute_values_snapshot_at_capture — mutates a list attribute after capture, asserts the two series keep distinct wire attributes.
  • test_failed_flushes_back_off_exponentially_capped — pins the [2, 4, 8, 16, 32, 64, 64]× delay schedule and the reset-on-success.
  • test_window_survives_failures_until_the_drop_limit / updated test_window_dropped_after_consecutive_failures — pin the drop-at-exactly-8 boundary from both sides.
  • test_hostile_metrics_config_does_not_raise_and_still_records — parameterized over five hostile configs, asserts the full public call chain records with defaults.

Full local validation: pytest posthog/test (1207 passed; the pre-existing test_integration_stop_reason failure reproduces on a clean checkout), ruff check/format --check, mypy | mypy-baseline filter (clean), python -W error -c "import posthog", make public_api_check (no public surface change).

📝 Checklist

  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

If releasing new changes

  • Ran sampo add to generate a changeset file (hand-written changeset, patch bump)

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

  • PostHog Code (Claude) session directed by @DanielVisca; work scoped to the three unresolved review threads on feat(metrics): add posthog.metrics count/gauge/histogram API (alpha) #739.
  • Chose deep-snapshot over restricting attributes to scalars (the reviewer offered both): Python's array-attribute support is already released and tested, so snapshotting fixes the aliasing bug without a behavior break.
  • Chose 8 attempts × capped-64× backoff over keeping the 3-attempt budget: with backoff spacing the retries out, a small budget would drop data ~2 minutes into an outage; 8 attempts covers ~21 minutes while the series cap bounds memory.

Created with PostHog Code

…alidation

Follow-ups from the #739 review:

- Deep-snapshot metric attributes at capture so mutating a nested
  list/dict afterwards cannot rewrite an already-keyed series' wire
  attributes.
- Retry failed flushes with exponential backoff (2x per consecutive
  failure, capped at 64x the flush interval, matching the shared JS
  logs policy) and drop the window after exactly 8 consecutive
  failures - the previous policy retried at the base cadence and
  dropped on the fourth failure while documenting three.
- Validate the nested metrics config and degrade to defaults with a
  warning, so client.metrics.count() keeps the client's no-throw
  contract even with a hostile config dict.

Generated-By: PostHog Code
Task-Id: f9941012-0ef9-4c6c-9813-34d84f8e7d02
@DanielVisca DanielVisca self-assigned this Jul 15, 2026
@github-actions

github-actions Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

posthog-python Compliance Report

Date: 2026-07-15 18:15:29 UTC
Duration: 338855ms

✅ All Tests Passed!

111/111 tests passed


Capture_V1 Tests

94/94 tests passed

View Details
Test Status Duration
Endpoint And Method.Targets V1 Endpoint 516ms
Endpoint And Method.Does Not Use Legacy Endpoints 1008ms
Required Headers.Has Authorization Bearer Header 1009ms
Required Headers.Has Content Type Json 1008ms
Required Headers.Has Posthog Sdk Info Format 1010ms
Required Headers.Has Posthog Attempt Header 1009ms
Required Headers.Has Posthog Request Id 1008ms
Required Headers.Has Posthog Request Timestamp 1009ms
Required Headers.Has User Agent 1009ms
Body Format.Body Has Created At And Batch 1008ms
Body Format.No Api Key In Body 1008ms
Body Format.No Sent At In Body 1008ms
Event Format.Event Has Required Root Fields 1008ms
Event Format.Event Uuid Is Valid 1008ms
Event Format.Event Timestamp Is Rfc3339 1009ms
Event Format.Distinct Id Is String 1009ms
Event Format.Distinct Id At Root Not Properties 1009ms
Event Format.Custom Properties Preserved 1008ms
Event Format.Set Properties Preserved 1010ms
Event Format.Set Once Properties Preserved 1008ms
Event Format.Groups Properties Preserved 1009ms
Event Format.Sdk Generates Uuid If Not Provided 1009ms
Event Format.Event Has Required Root Fields Batch 1012ms
Event Format.Event Uuid Is Valid Batch 1013ms
Event Format.Event Timestamp Is Rfc3339 Batch 1012ms
Event Format.Distinct Id Is String Batch 1012ms
Event Format.Distinct Id At Root Not Properties Batch 1012ms
Event Format.Custom Properties Preserved Batch 1013ms
Event Format.Set Properties Preserved Batch 1012ms
Event Format.Set Once Properties Preserved Batch 1013ms
Event Format.Groups Properties Preserved Batch 1012ms
Event Format.Sdk Generates Uuid If Not Provided Batch 1012ms
Batch Behavior.Multiple Events In Single Batch 1507ms
Batch Behavior.Batch Envelope Smoke 1014ms
Batch Behavior.Flush With No Events Sends Nothing 1004ms
Batch Behavior.Flush At Triggers Batch 1508ms
Batch Behavior.Created At Reflects Batch Creation Time 1011ms
Deduplication.Generates Unique Uuids 1507ms
Deduplication.Different Events Same Content Different Uuids 1507ms
Deduplication.Preserves Uuid On Retry 7516ms
Deduplication.Preserves Timestamp On Retry 7507ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry 7516ms
Deduplication.No Duplicate Events In Batch 1504ms
Header Behavior On Retry.Attempt Header Starts At One 1008ms
Header Behavior On Retry.Attempt Header Increments On Retry 14523ms
Header Behavior On Retry.Request Id Preserved On Retry 7509ms
Header Behavior On Retry.Different Requests Have Different Request Ids 3514ms
Header Behavior On Retry.Request Timestamp Changes On Retry 7514ms
Response Format Validation.Success Response Has Uuid Keyed Results 1006ms
Response Format Validation.Success Response Has Ok For Each Event 1507ms
Response Format Validation.Success No Retry After When All Ok 1507ms
Response Format Validation.Success Retry After Present When Retry Events 2511ms
Response Format Validation.Success No Retry After When Drop Only 1506ms
Response Format Validation.Response Echoes Request Id 1009ms
Retry Behavior.Retries On 408 7515ms
Retry Behavior.Retries On 500 7514ms
Retry Behavior.Retries On 503 9516ms
Retry Behavior.Retries On 504 7511ms
Retry Behavior.Retryable Errors Have Retry After 4506ms
Retry Behavior.Respects Retry After On Retryable Error 12520ms
Retry Behavior.Does Not Retry On 400 3502ms
Retry Behavior.Does Not Retry On 401 3509ms
Retry Behavior.Does Not Retry On 402 3508ms
Retry Behavior.Does Not Retry On 413 3506ms
Retry Behavior.Does Not Retry On 415 3508ms
Retry Behavior.Non Retryable Errors Have No Retry After 3507ms
Retry Behavior.Implements Backoff 23531ms
Retry Behavior.Max Retries Respected 23511ms
Partial Batch Handling.Handles 200 Full Success 3011ms
Partial Batch Handling.Handles 200 With All Ok 4508ms
Partial Batch Handling.Does Not Retry Dropped Events 4508ms
Partial Batch Handling.Does Not Retry Limited Events 4508ms
Partial Batch Handling.Prunes Ok Events On Partial Retry 7509ms
Partial Batch Handling.Prunes Dropped Events On Partial Retry 7516ms
Partial Batch Handling.Retries Only Retry Events From Partial 7512ms
Partial Batch Handling.Partial Retry Preserves Uuids 7512ms
Partial Batch Handling.Partial Retry Attempt Header Increments 7510ms
Partial Batch Handling.Partial Retry Request Id Preserved 7513ms
Partial Batch Handling.Respects Retry After On Partial 9512ms
Partial Batch Handling.Unknown Result Treated As Terminal 4506ms
Partial Batch Handling.Mixed Ok Drop Limited No Retry 4508ms
Compression.Sends Gzip Content Encoding 1007ms
Compression.No Content Encoding When Disabled 1009ms
Compression.Compressed Body Is Decompressible 1009ms
Error Handling.Does Not Retry On Unknown 4Xx 3509ms
Event Options.Cookieless Mode Override 1007ms
Event Options.Disable Skew Correction Override 1009ms
Event Options.Process Person Profile Override 1007ms
Event Options.Product Tour Id Override 1010ms
Event Options.Unset Options Omitted 1008ms
Event Options.Options Override In Batch 1013ms
Geoip And Historical Migration.Geoip Disable Injected Into Properties 1012ms
Geoip And Historical Migration.Historical Migration Set In Body 1008ms
Geoip And Historical Migration.Historical Migration Absent By Default 1009ms

Feature_Flags Tests

17/17 tests passed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id 1006ms
Request Payload.Flags Request Uses V2 Query Param 1006ms
Request Payload.Flags Request Hits Flags Path Not Decide 1007ms
Request Payload.Flags Request Omits Authorization Header 1006ms
Request Payload.Token In Flags Body Matches Init 1007ms
Request Payload.Groups Round Trip 1007ms
Request Payload.Groups Default To Empty Object 1006ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False 1006ms
Request Payload.Disable Geoip Omitted Defaults To False 1007ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key 1006ms
Request Lifecycle.No Flags Request On Init Alone 503ms
Request Lifecycle.No Flags Request On Normal Capture 1507ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests 1010ms
Request Lifecycle.Mock Response Value Is Returned To Caller 1003ms
Retry Behavior.Retries Flags On 502 1006ms
Retry Behavior.Retries Flags On 504 1007ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event 1508ms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant