fix(telemetry): keep OTLP receiver alive 6s after Shutdown to flush gateway batch exporter

[inference-gateway-maintainer]

Summary

Resolves #1004 - gateway spans at the end of a session were silently dropped because the OTLP receiver shut down before the gateway's batch exporter (5s default delay) could flush pending spans.

Changes

• internal/telemetry/recorder.go: Added a receiverGracePeriod package-level variable (6s, one batch window over the 5s default) and a time.Sleep(receiverGracePeriod) before closing the receiver server in Shutdown(), giving the gateway time to deliver its last batch.
• internal/telemetry/main_test.go: Override receiverGracePeriod = 0 in TestMain so tests return immediately instead of waiting 6s.

Design notes

• Package-level var is the single source of truth - no struct field needed (YAGNI). Tests override it directly.
• 6s margin is comfortable: the gateway's default batch delay is 5s, so 6s gives a full extra window without being excessive.
• The sleep only applies when a receiver server is active (r.recvSrv != nil), so sessions without OTLP tracing are unaffected.

Testing

• go build ./internal/telemetry/ passes
• go test ./internal/telemetry/ passes (grace period disabled in TestMain)
• task precommit:run passes (fmt, lint, mod-tidy, markdownlint)

[edenreich]

@infer Can you finalise the work? prepare this PR for review (update body and title). use /go /go-concurrency /ponytail:ponytail

[inference-gateway-maintainer]

Model: ollama_cloud/deepseek-v4-flash · View Job

Todos

• Read the current diff and understand existing changes
• Read issue [BUG] Gateway spans dropped at session end due to batch export race #1004 to understand the problem
• Fix compilation bugs: package-level var needs 'var', remove non-existent struct field, use package-level var in Shutdown
• Run pre-commit checks and fix any issues
• Update PR title and body for review

Pull Request

#1005

✅ Infer Result: Success

All tasks complete. Here's a summary of what was done:

Changes made

PR #1005 is now ready for review with updated title and body.

Fixes to the salvage commit

The original salvage commit (dfcdf34) had two compilation bugs that I fixed:

1. Missing var keyword on the package-level receiverGracePeriod declaration (Go requires var at package level)
2. Non-existent struct field receiverGracePeriod referenced in the Recorder struct literal - the struct has no such field

Ponytail simplification

Instead of adding a struct field (which would need a struct definition change and per-instance config nobody asked for), the fix uses the package-level variable directly in Shutdown():

• var receiverGracePeriod = 6 * time.Second - single source of truth
• time.Sleep(receiverGracePeriod) in Shutdown() - reads the package-level var
• receiverGracePeriod = 0 in TestMain - test override works directly

No struct field, no per-instance config (YAGNI), one fewer indirection.

Files changed (2 files, 14 insertions, 4 deletions)

• internal/telemetry/recorder.go - Added var receiverGracePeriod = 6 * time.Second and time.Sleep(receiverGracePeriod) before closing the receiver in Shutdown()
• internal/telemetry/main_test.go - Set receiverGracePeriod = 0 in TestMain to avoid the 6s delay in tests

Model: ollama_cloud/deepseek-v4-flash · Exit Code: 0 · Duration: 3m 1s

Tokens: 1,471,657 in · 9,461 out · 1,481,118 total (27 requests)

Tool calls: 32 total · 97% success rate

Traces

Session: 1785842619-476e7421

session (standard, success)                                        2m59s
├── chat ollama_cloud/deepseek-v4-flash                             2.6s
│   ╰── POST /v1/chat/completions [inference-gateway]               2.6s
│       ╰── HTTP POST [inference-gateway]                           2.6s
│           ╰── POST /proxy/:provider/*path [inference-gateway]     2.6s
│               ╰── HTTP POST [inference-gateway]                   2.6s
├── execute_tool TodoWrite call_x2ubxbjq                            73µs
├── chat ollama_cloud/deepseek-v4-flash                             1.6s
│   ╰── POST /v1/chat/completions [inference-gateway]               1.6s
│       ╰── HTTP POST [inference-gateway]                           1.6s
│           ╰── POST /proxy/:provider/*path [inference-gateway]     1.6s
│               ╰── HTTP POST [inference-gateway]                   1.6s
├── execute_tool Bash call_vvf1v1rd                                  9ms
├── execute_tool Bash call_xm2bw6bv                                  4ms
├── execute_tool Bash call_c1or6sjw                                  5ms
├── chat ollama_cloud/deepseek-v4-flash                             3.5s
│   ╰── POST /v1/chat/completions [inference-gateway]               3.5s
│       ╰── HTTP POST [inference-gateway]                           3.5s
│           ╰── POST /proxy/:provider/*path [inference-gateway]     3.5s
│               ╰── HTTP POST [inference-gateway]                   3.5s
├── execute_tool Bash call_jy0iravw                                785ms
├── execute_tool Read call_tlmiey9g                                433µs
├── execute_tool Read call_ax8wh8zt                                117µs
├── chat ollama_cloud/deepseek-v4-flash                            21.1s
│   ╰── POST /v1/chat/completions [inference-gateway]              21.1s
│       ╰── HTTP POST [inference-gateway]                          21.1s
│           ╰── POST /proxy/:provider/*path [inference-gateway]    21.1s
│               ╰── HTTP POST [inference-gateway]                  21.1s [error: *errors.errorString]
├── execute_tool TodoWrite call_39j4p398                            91µs
├── chat ollama_cloud/deepseek-v4-flash                             2.2s
│   ╰── POST /v1/chat/completions [inference-gateway]               2.2s
│       ╰── HTTP POST [inference-gateway]                           2.2s
│           ╰── POST /proxy/:provider/*path [inference-gateway]     2.2s
│               ╰── HTTP POST [inference-gateway]                   2.2s
├── execute_tool Bash call_rgfps6r6                                 4.4s [error: tool_error]
├── chat ollama_cloud/deepseek-v4-flash                            12.8s
│   ╰── POST /v1/chat/completions [inference-gateway]              12.8s
│       ╰── HTTP POST [inference-gateway]                          12.8s
│           ╰── POST /proxy/:provider/*path [inference-gateway]    12.8s
│               ╰── HTTP POST [inference-gateway]                  12.8s
├── execute_tool Read call_jydmrzs5                                185µs
├── chat ollama_cloud/deepseek-v4-flash                             3.6s
│   ╰── POST /v1/chat/completions [inference-gateway]               3.6s
│       ╰── HTTP POST [inference-gateway]                           3.6s
│           ╰── POST /proxy/:provider/*path [inference-gateway]     3.6s
│               ╰── HTTP POST [inference-gateway]                   3.6s
├── execute_tool Read call_2t2fe226                                271µs
├── execute_tool Read call_krzgu8vr                                193µs
├── chat ollama_cloud/deepseek-v4-flash                             4.8s
│   ╰── POST /v1/chat/completions [inference-gateway]               4.8s
│       ╰── HTTP POST [inference-gateway]                           4.8s
│           ╰── POST /proxy/:provider/*path [inference-gateway]     4.8s
│               ╰── HTTP POST [inference-gateway]                   4.8s
├── execute_tool TodoWrite call_e24lq8h4                            72µs
├── chat ollama_cloud/deepseek-v4-flash                             5.1s
│   ╰── POST /v1/chat/completions [inference-gateway]               5.1s
│       ╰── HTTP POST [inference-gateway]                           5.1s
│           ╰── POST /proxy/:provider/*path [inference-gateway]     5.1s
│               ╰── HTTP POST [inference-gateway]                   5.1s
├── chat ollama_cloud/deepseek-v4-flash                             2.5s
│   ╰── POST /v1/chat/completions [inference-gateway]               2.5s
│       ╰── HTTP POST [inference-gateway]                           2.5s
│           ╰── POST /proxy/:provider/*path [inference-gateway]     2.5s
│               ╰── HTTP POST [inference-gateway]                   2.5s
├── execute_tool Edit call_5yfec3y9                                408µs
├── chat ollama_cloud/deepseek-v4-flash                            14.9s
│   ╰── POST /v1/chat/completions [inference-gateway]              14.9s
│       ╰── HTTP POST [inference-gateway]                          14.9s
│           ╰── POST /proxy/:provider/*path [inference-gateway]    14.9s
│               ╰── HTTP POST [inference-gateway]                  14.9s
├── execute_tool Edit call_w5gv17k2                                478µs
├── chat ollama_cloud/deepseek-v4-flash                             3.4s
│   ╰── POST /v1/chat/completions [inference-gateway]               3.4s
│       ╰── HTTP POST [inference-gateway]                           3.4s
│           ╰── POST /proxy/:provider/*path [inference-gateway]     3.4s
│               ╰── HTTP POST [inference-gateway]                   3.4s
├── execute_tool Edit call_vh2j0hh1                                353µs
├── chat ollama_cloud/deepseek-v4-flash                             2.6s
│   ╰── POST /v1/chat/completions [inference-gateway]               2.6s
│       ╰── HTTP POST [inference-gateway]                           2.6s
│           ╰── POST /proxy/:provider/*path [inference-gateway]     2.6s
│               ╰── HTTP POST [inference-gateway]                   2.6s
├── execute_tool TodoWrite call_68xle3yx                            71µs
├── chat ollama_cloud/deepseek-v4-flash                             1.8s
│   ╰── POST /v1/chat/completions [inference-gateway]               1.8s
│       ╰── HTTP POST [inference-gateway]                           1.8s
│           ╰── POST /proxy/:provider/*path [inference-gateway]     1.8s
│               ╰── HTTP POST [inference-gateway]                   1.8s
├── execute_tool Bash call_kx8dh20n                                812ms
├── chat ollama_cloud/deepseek-v4-flash                             1.6s
│   ╰── POST /v1/chat/completions [inference-gateway]               1.6s
│       ╰── HTTP POST [inference-gateway]                           1.6s
│           ╰── POST /proxy/:provider/*path [inference-gateway]     1.6s
│               ╰── HTTP POST [inference-gateway]                   1.6s
├── execute_tool Bash call_6vr2igdw                                48.3s
├── chat ollama_cloud/deepseek-v4-flash                             2.3s
│   ╰── POST /v1/chat/completions [inference-gateway]               2.3s
│       ╰── HTTP POST [inference-gateway]                           2.3s
│           ╰── POST /proxy/:provider/*path [inference-gateway]     2.3s
│               ╰── HTTP POST [inference-gateway]                   2.3s
├── execute_tool Bash call_5ce2tk34                                  5ms
├── chat ollama_cloud/deepseek-v4-flash                             3.0s
│   ╰── POST /v1/chat/completions [inference-gateway]               3.0s
│       ╰── HTTP POST [inference-gateway]                           3.0s
│           ╰── POST /proxy/:provider/*path [inference-gateway]     3.0s
│               ╰── HTTP POST [inference-gateway]                   3.0s
├── execute_tool Bash call_788c31c7                                  7ms
├── chat ollama_cloud/deepseek-v4-flash                             2.2s
│   ╰── POST /v1/chat/completions [inference-gateway]               2.2s
│       ╰── HTTP POST [inference-gateway]                           2.2s
│           ╰── POST /proxy/:provider/*path [inference-gateway]     2.2s
│               ╰── HTTP POST [inference-gateway]                   2.2s
├── execute_tool Bash call_7f9irhvo                                  9ms
├── chat ollama_cloud/deepseek-v4-flash                             4.2s
│   ╰── POST /v1/chat/completions [inference-gateway]               4.2s
│       ╰── HTTP POST [inference-gateway]                           4.2s
│           ╰── POST /proxy/:provider/*path [inference-gateway]     4.2s
│               ╰── HTTP POST [inference-gateway]                   4.2s
├── chat ollama_cloud/deepseek-v4-flash                             3.0s
│   ╰── POST /v1/chat/completions [inference-gateway]               3.0s
│       ╰── HTTP POST [inference-gateway]                           3.0s
│           ╰── POST /proxy/:provider/*path [inference-gateway]     3.0s
│               ╰── HTTP POST [inference-gateway]                   3.0s
├── execute_tool Bash call_uhm0sew6                                 3.3s
├── chat ollama_cloud/deepseek-v4-flash                             1.5s
│   ╰── POST /v1/chat/completions [inference-gateway]               1.5s
│       ╰── HTTP POST [inference-gateway]                           1.5s
│           ╰── POST /proxy/:provider/*path [inference-gateway]     1.5s
│               ╰── HTTP POST [inference-gateway]                   1.5s
├── execute_tool Bash call_1ks8s0ok                                 1.3s
├── chat ollama_cloud/deepseek-v4-flash                             2.5s
│   ╰── POST /v1/chat/completions [inference-gateway]               2.5s
│       ╰── HTTP POST [inference-gateway]                           2.4s
│           ╰── POST /proxy/:provider/*path [inference-gateway]     2.4s
│               ╰── HTTP POST [inference-gateway]                   2.4s
├── execute_tool TodoWrite call_1qkol2wo                            72µs
├── chat ollama_cloud/deepseek-v4-flash                             3.5s
│   ╰── POST /v1/chat/completions [inference-gateway]               3.5s
│       ╰── HTTP POST [inference-gateway]                           3.5s
│           ╰── POST /proxy/:provider/*path [inference-gateway]     3.5s
│               ╰── HTTP POST [inference-gateway]                   3.5s
├── execute_tool Write call_eg7dxt2a                                 1ms
├── chat ollama_cloud/deepseek-v4-flash                             2.2s
│   ╰── POST /v1/chat/completions [inference-gateway]               2.2s
│       ╰── HTTP POST [inference-gateway]                           2.2s
│           ╰── POST /proxy/:provider/*path [inference-gateway]     2.2s
│               ╰── HTTP POST [inference-gateway]                   2.2s
├── execute_tool Bash call_idw3a2pc                                 1.8s
├── chat ollama_cloud/deepseek-v4-flash                             1.7s
│   ╰── POST /v1/chat/completions [inference-gateway]               1.7s
│       ╰── HTTP POST [inference-gateway]                           1.7s
│           ╰── POST /proxy/:provider/*path [inference-gateway]     1.7s
│               ╰── HTTP POST [inference-gateway]                   1.7s
├── execute_tool Bash call_3lpj7y44                                  4ms
├── execute_tool Bash call_hsi20j5b                                  9ms
├── chat ollama_cloud/deepseek-v4-flash                             2.4s
│   ╰── POST /v1/chat/completions [inference-gateway]               2.4s
│       ╰── HTTP POST [inference-gateway]                           2.4s
│           ╰── POST /proxy/:provider/*path [inference-gateway]     2.4s
│               ╰── HTTP POST [inference-gateway]                   2.4s
├── execute_tool TodoWrite call_wn6qkcmy                            69µs
╰── chat ollama_cloud/deepseek-v4-flash                             4.1s

Stats

Tool Calls

╭───────────┬───────┬───────┬────────╮
│ Tool      │ Calls │ Fail% │ Avg    │
├───────────┼───────┼───────┼────────┤
│ Bash      │ 15    │ 7%    │ 4053ms │
│ TodoWrite │ 6     │ 0%    │ 0ms    │
│ Read      │ 5     │ 0%    │ 0ms    │
│ Edit      │ 3     │ 0%    │ 0ms    │
│ Write     │ 1     │ 0%    │ 1ms    │
╰───────────┴───────┴───────┴────────╯

Token Usage

╭────────────────────────────────┬─────────┬────────┬────────────┬─────────┬──────╮
│ Model                          │ Prompt  │ Cached │ Completion │ Total   │ Cost │
├────────────────────────────────┼─────────┼────────┼────────────┼─────────┼──────┤
│ ollama_cloud/deepseek-v4-flash │ 1471657 │ 0      │ 9461       │ 1481118 │ -    │
╰────────────────────────────────┴─────────┴────────┴────────────┴─────────┴──────╯

Sessions

╭───────────┬──────────┬──────────╮
│ Execution │ Mode     │ Sessions │
├───────────┼──────────┼──────────┤
│ headless  │ standard │ 1        │
╰───────────┴──────────┴──────────╯

⚠️ 1 log(s)

• Bash call_rgfps6r6:

  exit status 1: # github.com/inference-gateway/cli/internal/telemetry
  internal/telemetry/recorder.go:49:1: syntax error: non-declaration statement outside function body
  

Triggered by edenreich · Infer Action · OSS

[inference-gateway-releaser]

🎉 This PR is included in version 0.160.1 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀