Skip to content

test: fix concurrent start test assertions#1094

Open
nanookclaw wants to merge 2 commits into
temporalio:mainfrom
nanookclaw:fix/concurrent-starts-test-harness
Open

test: fix concurrent start test assertions#1094
nanookclaw wants to merge 2 commits into
temporalio:mainfrom
nanookclaw:fix/concurrent-starts-test-harness

Conversation

@nanookclaw

Copy link
Copy Markdown

Summary

Fixes #581 by removing testing.T assertions from the worker goroutines in TestServer_StartDev_ConcurrentStarts. Each concurrent start now returns an error, and the main test goroutine collects those errors and reports them with require.NoError.

The test still starts 40 dev server instances with a fixed worker pool of 6 concurrent executions. The replacement command helper runs server start-dev with a cancellable context and captured stdout/stderr, so failures from early command exit or cleanup are preserved in the final assertion message without calling FailNow-style methods outside the test goroutine.

Verification

  • gofmt -w internal/temporalcli/commands.server_test.go
  • git diff --check
  • go test ./internal/temporalcli -run '^TestServer_StartDev_ConcurrentStarts$' -count=1

@nanookclaw nanookclaw requested a review from a team as a code owner June 21, 2026 04:27
@CLAassistant

CLAassistant commented Jun 21, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors TestServer_StartDev_ConcurrentStarts to avoid calling testing.T/require assertions from worker goroutines by having each concurrent start return an error, then asserting in the main test goroutine. This aligns with Go’s testing constraints and should reduce flakiness/misreporting in the concurrent dev-server start integration test.

Changes:

  • Reworked the concurrent-start worker logic to return errors from each run and aggregate them in the main goroutine.
  • Replaced the command harness usage inside worker goroutines with a small helper that executes temporalcli.Execute using captured stdout/stderr.
  • Updated the concurrency level to a fixed worker pool of 6 and improved error reporting to include command output.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

h := NewCommandHarness(t)
defer h.Close()
startOne := func() error {
ctx, cancel := context.WithCancel(context.Background())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nanookclaw I like the approach. let's use t.Context() instead of context.Background()

}
}

func executeConcurrentStartCommand(ctx context.Context, args ...string) *CommandResult {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's add func (h *CommandHarness) ExecuteWithContext(ctx context.Context, args ...string) *CommandResult {

so we can keep using the harness

var cl client.Client
h.EventuallyWithT(func(t *assert.CollectT) {
var lastDialErr error
for start := time.Now(); time.Since(start) < 3*time.Second; {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about something like this:

 ticker := time.NewTicker(200 * time.Millisecond)
  defer ticker.Stop()
  timeout := time.NewTimer(3 * time.Second)
  defer timeout.Stop()

  outer:
  for {
      select {
      case res := <-resCh:
          return concurrentStartCommandResultError("got early server result", res)
      case <-timeout.C:
          break outer
      case <-ticker.C:
          var err error
          cl, err = client.Dial(client.Options{HostPort: "127.0.0.1:" + port})
          if err == nil {
              break outer
          }
          lastDialErr = err
      }
  }

@nanookclaw nanookclaw force-pushed the fix/concurrent-starts-test-harness branch from 64ade53 to ce9c1a5 Compare June 22, 2026 16:25
@nanookclaw

Copy link
Copy Markdown
Author

Updated in ce9c1a5 to address the review comments:

  • added CommandHarness.ExecuteWithContext(ctx, args...) while preserving existing Execute behavior
  • changed the concurrent start test to use t.Context() and the harness
  • replaced the readiness sleep loop with a ticker/timer select
  • removed the bespoke concurrent-start command helper

Local verification:

  • go test ./internal/temporalcli -run '^$'
  • go test ./internal/temporalcli -run '^TestServer_StartDev_ConcurrentStarts$' -count=1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Dev Server concurrent execution test is incorrect and flaky

4 participants