feat(metrics): add posthog.metrics count/gauge/histogram API (alpha)#739
Conversation
Port of the posthog-js core metrics client: statsd-style pre-aggregation (counts sum, gauges keep last, histograms bucket with the OTel default bounds), one OTLP/JSON data point per series per flush window, delta temporality, posted to /i/v1/metrics with the project token. Thread-safe; a daemon timer flushes every 10s and shutdown() drains the window. Includes the cardinality guardrail (max_series_per_flush, default 1000), the type-collision warning, transient-failure window merge-back, and the same resource-attribute layering (SDK keys over user keys) as posthog-js. Generated-By: PostHog Code Task-Id: 0d65d94c-c18a-4d84-93f6-18eb54876551
posthog-python Compliance ReportDate: 2026-07-14 21:55:16 UTC ✅ All Tests Passed!111/111 tests passed Capture_V1 Tests✅ 94/94 tests passed View Details
Feature_Flags Tests✅ 17/17 tests passed View Details
|
Crash safety (a telemetry SDK must never raise into the host app): the series key is now JSON-based like the JS core's seriesKey, so list/dict attribute values and mixed-type keys aggregate instead of raising TypeError; before_send returns are validated (non-dict dropped, unknown metric types dropped instead of folding as histograms); and a catch-all guard drops-with-warning anything that still escapes. Fork safety: a forked child inherits the parent's window and a timer handle whose thread doesn't exist in the child — captures now detect the PID change, drop the inherited window (avoiding duplication), and re-arm. Failure handling: retry-later flushes log a warning and are bounded by a 3-consecutive-failure budget (then dropped loudly), merge-back respects the series cap so an outage with attribute churn can't grow memory without bound, and the send goes through the shared pooled session with a gzipped body and a URL-encoded token. Encoding parity with posthog-js: non-finite floats emit proto3 literals (Infinity/-Infinity/NaN), integral floats emit intValue, None-valued attributes are stripped before keying, and bool/int values stay distinct series via the JSON key. Generated-By: PostHog Code Task-Id: 0d65d94c-c18a-4d84-93f6-18eb54876551
|
Reviews (1): Last reviewed commit: "fix(metrics): harden the capture path an..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Typed extraction after the before_send isinstance narrow (dict[str, Any] + closed defaults the validation below drops), RequestException added to the vendored requests stub as the base the real hierarchy has, and the public API snapshot regenerated for the new Client.metrics surface. Generated-By: PostHog Code Task-Id: 0d65d94c-c18a-4d84-93f6-18eb54876551
Series identity already str()s keys, but the payload emitted the raw object — a numeric key violates OTLP's string KeyValue.key and strict decoders reject or drop the attribute. Generated-By: PostHog Code Task-Id: 0d65d94c-c18a-4d84-93f6-18eb54876551
Address review of the metrics capture surface: - _do_flush discards the window without transmitting when client.send is False, mirroring event capture (record locally, send nothing). - _reinit_after_fork replaces the metrics locks (_metrics_lock, _lock, _flush_lock) and drops the inherited window/timer in the forked child, so a child can't deadlock on a lock held by a vanished parent thread. - Module-level shutdown() delegates to Client.shutdown() so the final metrics window is flushed for apps using the global SDK lifecycle. Generated-By: PostHog Code Task-Id: 9606289a-ed0e-4b1b-828e-3b7f405d31e4
DanielVisca
force-pushed
the
posthog-code/metrics-python
branch
from
7bbb418 to
04d9d8b
Compare
dustinbyrne
left a comment
dustinbyrne
left a comment
There was a problem hiding this comment.
Human-driven, agent-assisted follow-up review.
| self.unit = unit | ||
| # Snapshot: the series key was computed from these values, so a caller | ||
| # mutating the dict after capture must not change the stored series. | ||
| self.attributes = dict(attributes) if attributes else None |
There was a problem hiding this comment.
[P1] Snapshot nested attribute values or enforce the scalar contract
_SeriesState only shallow-copies the attributes dictionary. Python currently treats nested values as supported: _series_key() explicitly handles lists/dicts, _to_otlp_any_value() encodes arrays, and the tests cover list attributes.
This lets the stored attributes diverge from the series key. For example:
- Record with
attributes={"tags": ["a"]}. - Mutate the list to
["a", "b"]. - Record again and flush.
The current payload has two points, both with tags=["a", "b"] and value 1, although the first series key was created for ["a"].
Browser and Node restrict metric attributes to scalar values, making their shallow copy sufficient. Please either deeply snapshot/normalize Python's supported nested values, or align Python with the other two SDKs by validating attributes as scalars and dropping nested values. Add a regression that mutates a nested value after the first capture and verifies stable wire attributes.
| if outcome == "retry-later": | ||
| with self._lock: | ||
| self._consecutive_send_failures += 1 | ||
| if self._consecutive_send_failures > _MAX_CONSECUTIVE_SEND_FAILURES: |
There was a problem hiding this comment.
[P2 parity] Define a consistent bounded retry policy across metrics SDKs
Python retries at the fixed flush interval and drops the window on the fourth failure. Browser and Node also use a fixed metrics flush interval, but retain and retry the window forever; neither applies exponential backoff across failed metrics flushes. Node only adds fixed-delay transport retries within each attempt.
Retrying forever at a fixed cadence does not seem like the right behavior to copy. Can we choose an explicit cross-SDK policy—ideally capped exponential backoff plus a documented attempt or age limit—and pin it with tests? The shared JS logs implementation already uses exponential backoff capped at 64× the base interval and may be a useful precedent.
At minimum, Python's release note and implementation need reconciling: it says the window is dropped after three failed flushes, while this > 3 condition drops on the fourth.
| if self._metrics is None: | ||
| with self._metrics_lock: | ||
| if self._metrics is None: | ||
| self._metrics = PostHogMetrics(self, self._metrics_config) |
There was a problem hiding this comment.
[P1] Keep lazy metrics initialization inside the client's no-throw contract
Most public Client methods use @no_throw, but client.metrics constructs PostHogMetrics before the guarded recording path is reached. Because the nested config is typed only as dict, a type checker accepts:
Client("phc_...", metrics={"resource_attributes": "bad"})and client.metrics.count(...) then raises AttributeError into application code.
Please give this path the same no-throw semantics as the rest of the client: validate nested configuration and fall back to defaults or a valid no-op metrics instance when initialization fails. Decorating the property with @no_throw() while returning None would not be sufficient, because the chained .count(...) would still throw. Add a regression asserting the complete public call does not raise.
dustinbyrne
left a comment
dustinbyrne
left a comment
There was a problem hiding this comment.
approving to unblock but there's some useful considerations in here
2 participants
What this is
The Python leg of the
posthog.metricsSDK surface (Metrics docs) — a faithful port of the posthog-js core client (posthog-js #4115, node wiring in #4117), so backend Python services can record metrics with zero OpenTelemetry setup:Design (mirrors posthog-js exactly, adapted to Python)
count()calls is one data point on the wire. Delta temporality throughout, so restarts need no cross-window state.ExportMetricsServiceRequestto{host}/i/v1/metrics?token=...— nano timestamps as decimal strings but histogramcount/bucketCountsas plain JSON numbers (string-encoded u64s are silently dropped upstream, opentelemetry-rust#3328). SameDEFAULT_HISTOGRAM_BOUNDSas JS.threading.Timerarmed on first capture (10s default).shutdown()flushes and resets.max_series_per_flushcardinality cap (default 1000, warn once per window, existing series keep folding), type-collision warning, monotonic-counter validation, finite-value validation,before_sendhook.posthog/metrics_capture.py(notmetrics.py— that name would shadow a future module-levelposthog.metricsaccessor and matches theexception_capture.pyhouse naming).How did you test this code?
TDD — the test file was written first and observed failing at import, then all green:
posthog/test/test_metrics.py— 14 tests, each pinning a distinct contract: count burst → one delta+monotonic data point; gauge keeps last (and has nostartTimeUnixNano, matching JS); the full histogram wire shape (the opentelemetry-rust#3328 encoding pin); series identity split by attributes but not by attribute order; OTLP value encoding (including the Python-specific bool-before-int trap —boolis anintsubclass and would silently encodeTrueasintValue: 1); cardinality cap drops new series but keeps folding existing ones; parameterized invalid-sample drops; disabled client no-ops; resource-attribute layering (user keys can't spooftelemetry.sdk.*); transient-failure merge-back (a 503'd window's counts sum into the next flush); shutdown flush.posthog/test/test_client.py— 145 passed (I touchedClient.__init__andshutdown).ruff check/ruff formatclean.check_public_api.py --write(griffe isn't installable in my env — PEP 668); CI should regenerate/check the snapshot, and the newClient.metricsproperty will need it. Also no live end-to-end send from this environment — the wire shape is byte-identical to posthog-js's, which was validated end-to-end against ingestion during the metrics alpha.🤖 Agent context
Autonomy: Human-driven (agent-assisted) — Daniel directed this; assigned as DRI.
I (Claude) wrote this by porting
packages/core/src/metrics/from posthog-js (read in full: index.ts, metrics-utils.ts, config.ts, the value encoder in logs-utils.ts, and the_sendMetricsBatchtransport for the URL/auth/outcome contract). Deliberate deviations from the JS original:flush_intervalin seconds (Python client idiom, vsflushIntervalMs), thread-safety via locks (JS is single-threaded), nodrainWindow()(browser-unload concern), and plain-requeststransport with the same ok/too-large/retry-later/fatal outcome classification instead of the fetch retry stack. Part of the metrics-SDK track: js ✅ merged, node in review, python (this PR).