Skip to content

Fix sampling log probe with capture expressions#11571

Open
jpbempel wants to merge 2 commits into
masterfrom
jpbempel/fix-cap-expr-sampling
Open

Fix sampling log probe with capture expressions#11571
jpbempel wants to merge 2 commits into
masterfrom
jpbempel/fix-cap-expr-sampling

Conversation

@jpbempel
Copy link
Copy Markdown
Member

@jpbempel jpbempel commented Jun 4, 2026

What Does This Do

A log probe with CaptureExpressions has captureSnapshot=false but this is not a log template so the sampling must be 1/s like a regular snapshot

Motivation

Additional Notes

Contributor Checklist

  • Format the title according to the contribution guidelines
  • Assign the type: and (comp: or inst:) labels in addition to any other useful labels
  • Avoid using close, fix, or any linking keywords when referencing an issue
    Use solves instead, and assign the PR milestone to the issue
  • Update the CODEOWNERS file on source file addition, migration, or deletion
  • Update public documentation with any new configuration flags or behaviors
  • Add your completed PR to the merge queue by commenting /merge. You can also:
    • Customize the commit message associated with the merge with /merge --commit-message "..."
    • Remove your PR from the merge queue with /merge -c
    • Skip all merge queue checks with /merge -f --reason "reason"; please use this judiciously, as some checks do not run at the PR-level (note: the PR still needs to be mergeable, this will only skip the pre-merge build)
    • Get more information in this doc

Jira ticket: [DEBUG-5730]

A log probe with CaptureExpressions has captureSnapshot=false but
this is not a log template so the sampling must be 1/s like a regular
snapshot
@jpbempel jpbempel requested a review from a team as a code owner June 4, 2026 19:43
@jpbempel jpbempel requested review from evanchooly and removed request for a team June 4, 2026 19:43
@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented Jun 4, 2026

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@jpbempel jpbempel added type: bug Bug report and fix comp: debugger Dynamic Instrumentation labels Jun 4, 2026
@datadog-datadog-prod-us1

This comment has been minimized.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 09f04eb975

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

if (!hasCondition()) {
// we are sampling here to avoid creating CapturedContext when the sampling result is negative
return ProbeRateLimiter.tryProbe(sampler, isCaptureSnapshot());
return ProbeRateLimiter.tryProbe(sampler, isFullSnapshot());
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use snapshot per-probe rate for capture expressions

For a capture-expression log probe without explicit sampling, this now selects the snapshot global limiter, but initSamplers() still builds sampler with DEFAULT_LOG_RATE whenever captureSnapshot(false) is set. In that scenario the per-probe limiter remains 5000/s even though capture expressions are committed as full snapshots, so one probe can overproduce until the global snapshot budget is exhausted; base the default sampler rate on isFullSnapshot() as well.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented Jun 4, 2026

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.01 s 13.99 s [-1.0%; +1.2%] (no difference)
startup:insecure-bank:tracing:Agent 12.88 s 13.04 s [-2.2%; -0.2%] (maybe better)
startup:petclinic:appsec:Agent 16.38 s 16.37 s [-1.1%; +1.2%] (no difference)
startup:petclinic:iast:Agent 15.73 s 16.47 s [-12.9%; +3.9%] (unstable)
startup:petclinic:profiling:Agent 15.56 s 16.41 s [-13.5%; +3.1%] (unstable)
startup:petclinic:tracing:Agent 15.10 s 15.75 s [-12.5%; +4.4%] (unstable)

Commit: d0af61a5 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Jun 4, 2026

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1780604002 1780604364
end_time 2026-06-04T20:14:52 2026-06-04T20:20:55
git_branch master jpbempel/fix-cap-expr-sampling
git_commit_sha bacdc33 d0af61a
start_time 2026-06-04T20:13:23 2026-06-04T20:19:25
See matching parameters
Baseline Candidate
ci_job_id 1744490799 1744490799
ci_pipeline_id 117171485 117171485
cpu_model Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
git_commit_date 1780603378 1780603378

Summary

Found 5 performance improvements and 0 performance regressions! Performance is the same for 4 metrics, 6 unstable metrics.

scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:loop better
[-805.884µs; -606.563µs] or [-8.368%; -6.299%]
better
[-864.532µs; -648.204µs] or [-8.847%; -6.633%]
better
[-828.707µs; -622.317µs] or [-8.349%; -6.270%]
better
[-832.984µs; -618.771µs] or [-8.235%; -6.118%]
better
[+5.112op/s; +9.084op/s] or [+5.061%; +8.993%]
See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-51.627µs; +44.402µs] or [-15.897%; +13.673%]
unstable
[-63.734µs; +59.658µs] or [-17.158%; +16.060%]
unstable
[-78.661µs; +70.580µs] or [-20.049%; +17.989%]
unstable
[-456.969µs; +395.731µs] or [-32.478%; +28.126%]
same
scenario:basic same same same unstable
[-11.827µs; +176.635µs] or [-1.085%; +16.198%]
unstable
[-163.139op/s; +163.139op/s] or [-6.852%; +6.852%]
Request duration reports for reports
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (371.462 µs) : 308, 435
.   : milestone, 371,
basic (309.385 µs) : 301, 317
.   : milestone, 309,
loop (9.772 ms) : 9630, 9914
.   : milestone, 9772,
section candidate
noprobe (369.424 µs) : 317, 422
.   : milestone, 369,
basic (311.772 µs) : 303, 320
.   : milestone, 312,
loop (9.016 ms) : 9008, 9023
.   : milestone, 9016,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 371.462 µs [307.707 µs, 435.216 µs]
basic 309.385 µs [301.35 µs, 317.42 µs]
loop 9.772 ms [9.63 ms, 9.914 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 369.424 µs [316.516 µs, 422.331 µs]
basic 311.772 µs [303.292 µs, 320.253 µs]
loop 9.016 ms [9.008 ms, 9.023 ms]

@jpbempel jpbempel added this pull request to the merge queue Jun 5, 2026
@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented Jun 5, 2026

/merge

@gh-worker-devflow-routing-ef8351
Copy link
Copy Markdown

gh-worker-devflow-routing-ef8351 Bot commented Jun 5, 2026

View all feedbacks in Devflow UI.

2026-06-05 04:52:48 UTC ℹ️ Start processing command /merge


2026-06-05 04:52:53 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).


2026-06-05 06:53:31 UTCMergeQueue: The build pipeline has timeout

The merge request has been interrupted because the build 5920116106124784514 took longer than expected. The current limit for the base branch 'master' is 120 minutes.

@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jun 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: debugger Dynamic Instrumentation type: bug Bug report and fix

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants