Skip to content

fix: scan AGENTS.md/CLAUDE.md for Go test commands (#470)#487

Open
aviavraham wants to merge 1 commit into
ambient-code:mainfrom
aviavraham:470-go-test-agents-md
Open

fix: scan AGENTS.md/CLAUDE.md for Go test commands (#470)#487
aviavraham wants to merge 1 commit into
ambient-code:mainfrom
aviavraham:470-go-test-agents-md

Conversation

@aviavraham
Copy link
Copy Markdown
Contributor

@aviavraham aviavraham commented May 31, 2026

Summary

Fixes #470.

  • AGENTS.md/CLAUDE.md scanned: _read_go_build_files() now includes these files alongside Makefile, Taskfile.yml, README.md, and CI workflows
  • Broader test command regex: The main check now matches make test and ginkgo in addition to go test / $(GO) test, so repos using build wrappers or the Ginkgo test framework are recognized

The root cause was two-fold: AGENTS.md wasn't in the scan list for the main 20-point "test command found" check, and the regex only matched literal go test invocations — missing make test wrappers and Ginkgo.

Test plan

  • New test: make test in AGENTS.md detected as a GO test command
  • All 48 Go assessor tests pass (no regressions)
  • Lint clean

Summary by CodeRabbit

  • New Features
    • Improved Go test-command detection to recognize additional patterns including $(GO) go test, make test, and ginkgo
    • Extended project analysis to scan additional documentation files (AGENTS.md, CLAUDE.md) for Go-related configuration and commands

_read_go_build_files() now includes AGENTS.md and CLAUDE.md in its file
scan. The test command regex also matches `make test` and `ginkgo`, not
just `go test`, so repos using build wrappers or Ginkgo are recognized.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 31, 2026

📝 Walkthrough

Walkthrough

The PR expands Go test-command detection to recognize $(GO) go test, make test, and ginkgo patterns, and adds AGENTS.md and CLAUDE.md to the files scanned for Go-related commands. A new regression test validates detection of test commands documented in AGENTS.md.

Changes

Go test command detection expansion for AGENTS.md

Layer / File(s) Summary
Test command detection patterns
src/agentready/assessors/testing.py
The regex for Go test-command detection broadens to match $(GO) go test, make test, and ginkgo alongside plain go test, with updated evidence messaging.
Go build file sourcing
src/agentready/assessors/testing.py
Scanning expands to AGENTS.md and CLAUDE.md files, enabling the assessor to find test commands documented in project guides.
Regression test
tests/unit/test_assessors_go.py
New test confirms detection of Go test commands in AGENTS.md with minimum score threshold enforcement.

Possibly related issues

  • #470: Directly addresses the bug where commands documented in AGENTS.md (e.g., make test) are not recognized during assessment.

Possibly related PRs

  • ambient-code/agentready#481: Aligns with TestExecutionAssessor enhancements to scan AGENTS.md and CLAUDE.md for documented Go test commands.
  • ambient-code/agentready#412: Related Go monorepo support work that identified gaps in _assess_go_coverage test-command pattern matching.

Suggested labels

released

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title follows Conventional Commits format (fix: description) and clearly describes the main change: scanning AGENTS.md/CLAUDE.md for Go test commands.
Linked Issues check ✅ Passed The PR fully addresses issue #470 by adding AGENTS.md and CLAUDE.md to scanned files and expanding test-command regex to match make test and ginkgo patterns.
Out of Scope Changes check ✅ Passed All changes are scoped to fixing the test-command detection issue: regex expansion and file-list expansion in the assessor, plus a regression test.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
✨ Simplify code
  • Create PR with simplified code

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Copy Markdown
Contributor

📈 Test Coverage Report

Branch Coverage
This PR 73.4%
Main 73.4%
Diff ✅ +0%

Coverage calculated from unit tests only

@aviavraham
Copy link
Copy Markdown
Contributor Author

Verification against reported repo

Ran the updated assessor against konflux-ci/multi-platform-controller (from the issue):

Status:  pass
Score:   100.0
Value:   configured
Evidence:
  - Go test files found (*_test.go)
  - Go test command found in project files
  - Coverage configuration found
  - Race detector enabled (-race flag)
  - Test command documented in AGENTS.md
  - Test organization: Makefile targets (test-e2e)

Previously reported "No 'go test' command found in Makefile/CI/README" because:

  1. AGENTS.md wasn't scanned by the main check
  2. The Makefile uses $(GINKGO) instead of go test, which the old regex didn't match

@aviavraham aviavraham marked this pull request as ready for review May 31, 2026 15:23
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/agentready/assessors/testing.py`:
- Around line 642-643: Update the monorepo subdirectory scan to include
AGENTS.md and CLAUDE.md the same way README.md is added for Go module
subdirectories: extend the logic that currently appends repository.path /
"AGENTS.md" and repository.path / "CLAUDE.md" (and the code that adds Makefile,
Taskfile.yml, README.md in the Go module loop) so those two files are also
checked/collected from each submodule directory rather than only at repo root;
locate the block that builds the per-submodule file list (where README.md is
appended) and add entries for "AGENTS.md" and "CLAUDE.md" to that list.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: b045a269-80a1-4c0f-9090-bffcfacbfa90

📥 Commits

Reviewing files that changed from the base of the PR and between 6470c9f and 3cc31e3.

📒 Files selected for processing (2)
  • src/agentready/assessors/testing.py
  • tests/unit/test_assessors_go.py

Comment on lines +642 to +643
repository.path / "AGENTS.md",
repository.path / "CLAUDE.md",
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial | 💤 Low value

Consider adding AGENTS.md/CLAUDE.md to monorepo subdirectory scan for consistency.

Lines 650-655 add Makefile, Taskfile.yml, and README.md from Go module subdirectories, but AGENTS.md and CLAUDE.md are only checked at the repository root. This is likely intentional (these docs typically live at root), but creates a minor inconsistency with how README.md is handled.

♻️ Optional consistency improvement
             files_to_check.extend(
                 [
                     module_root / "Makefile",
                     module_root / "Taskfile.yml",
                     module_root / "README.md",
+                    module_root / "AGENTS.md",
+                    module_root / "CLAUDE.md",
                 ]
             )
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
repository.path / "AGENTS.md",
repository.path / "CLAUDE.md",
files_to_check.extend(
[
module_root / "Makefile",
module_root / "Taskfile.yml",
module_root / "README.md",
module_root / "AGENTS.md",
module_root / "CLAUDE.md",
]
)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/agentready/assessors/testing.py` around lines 642 - 643, Update the
monorepo subdirectory scan to include AGENTS.md and CLAUDE.md the same way
README.md is added for Go module subdirectories: extend the logic that currently
appends repository.path / "AGENTS.md" and repository.path / "CLAUDE.md" (and the
code that adds Makefile, Taskfile.yml, README.md in the Go module loop) so those
two files are also checked/collected from each submodule directory rather than
only at repo root; locate the block that builds the per-submodule file list
(where README.md is appended) and add entries for "AGENTS.md" and "CLAUDE.md" to
that list.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Commands documented in AGENTS.md file are not parsed

1 participant