Skip to content

feat: org-level report - team-wide agent usage, cost, and productivity digest for engineering leads #214

Description

@Siddhant-K-code

Problem

Engineering leads and CTOs have no visibility into what their teams are building with AI agents, what it costs at the org level, or where the waste is. Individual developers see their own session costs. Nobody sees the aggregate.

The questions that go unanswered:

  • Which teams are using agents most? Most efficiently?
  • What is the org spending on AI agents per month, broken down by team and project?
  • Where are the debugging loops that are burning budget?
  • What are agents actually being used for - feature work, tests, refactoring?

Proposed solution

agent-strace org-report - aggregate sessions across team members and produce a structured digest for engineering leads.

# Generate org report for the current month
agent-strace org-report --month 2026-06

# Export as HTML for sharing with leadership
agent-strace org-report --month 2026-06 --format html --output june-report.html

# Scope to a specific team (by directory or tag)
agent-strace org-report --month 2026-06 --team backend

How sessions are collected

Two modes:

  1. Hosted collector (recommended for teams): all sessions push to AGENT_STRACE_ENDPOINT; org-report queries the collector API filtered by team/org
  2. Shared directory (self-hosted): sessions stored in a shared NFS/S3 path; org-report reads from --trace-dir

Report sections

Executive summary

June 2026 - Engineering AI Agent Report
────────────────────────────────────────
Total spend:        $1,847.22
Sessions:           2,341
Active engineers:   18 / 24
Tasks completed:    89.3%
Avg cost/session:   $0.79

Team breakdown

Team         Sessions   Cost      Avg cost   Efficiency
─────────────────────────────────────────────────────────
backend         841    $712.44    $0.85       good
frontend        634    $398.21    $0.63       good
platform        412    $501.18    $1.22       ⚠️ high
data             89     $98.11    $1.10       ⚠️ high
mobile          365    $137.28    $0.38       excellent

Anomaly callouts

⚠️  platform team: avg cost $1.22 vs org median $0.79 (+54%)
    -> 3 engineers account for 71% of platform spend
    -> Top pattern: debugging loops (tool-loop lint violations: 142)

⚠️  data team: 31% of sessions ended in timeout
    -> Likely cause: long-running data processing tasks hitting budget ceiling

Task type breakdown (from behavioral fingerprint classification)

Task type          Sessions   Avg cost   Sweet spot
────────────────────────────────────────────────────
Unit tests            412      $0.21      ✅ efficient
Bug fixes             389      $0.68      ✅ efficient
Refactoring           298      $0.94      ✅ efficient
Architecture          187      $2.41      ⚠️ above sweet spot
Code review           156      $0.31      ✅ efficient
Debugging             899      $1.18      ⚠️ above sweet spot

Implementation notes

  • Builds on existing team cost attribution (v0.8x) and behavioral fingerprint (v0.8x) commands
  • Anomaly detection uses the same baseline logic as watch anomaly detection
  • HTML output is a self-contained file (no CDN, no external deps) - same pattern as replay --format html
  • No PII in the report: engineer names are optional, can be anonymized with --anonymize

Acceptance criteria

  • agent-strace org-report aggregates sessions from hosted collector or shared trace dir
  • Report includes: total spend, session count, team breakdown, task type breakdown, anomaly callouts
  • --format html produces a self-contained shareable HTML file
  • --format json for programmatic consumption
  • --anonymize replaces engineer identifiers with anonymous labels
  • --team filters to a specific team tag or directory scope
  • Anomaly callouts surface teams/engineers significantly above org median cost
  • Docs: setup guide for team-wide collection (hosted collector + shared dir modes)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions