Skip to content

dispatch drops spec.repo when calling fleet.spawn — relay placement can't filter by repo, silently runs on the orchestrator's own node #84

Description

@khaliqgant

Summary

factory dispatch --backend relay drops the repo it already knows when calling fleet.spawn, so relay placement cannot filter candidate nodes by repo. Placement then selects the first live node advertising the capability — which in a mixed fleet is frequently the orchestrator's own machine. The loop still produces a real PR, so this fails silently and looks like a pass.

Found while proving cross-node placement for AgentWorkforce/cloud#2655 (hosted factory = control plane + any-surface nodes).

The bug

src/orchestrator/factory.ts:2484 (origin/main @ 99611d8, published in 0.1.19). The spec has reporouteImplementerSpec/routeReviewerSpec both set repo: route.repo — but the spawn call omits it:

result = await this.#fleet.spawn({
  name: spec.name,
  capability: spec.capability,
  node: spec.node ?? 'self',
  // repo: spec.repo,   <-- MISSING
  task: spec.task,
  ...
  cwd: spec.clonePath,
})

RelayFleetClient.spawn already forwards it when present (src/fleet/relay-fleet-client.ts): ...(input.repo ? { repo: input.repo } : {}). So the plumbing exists on both sides; only the call site is missing the field. Same omission at the restart spawn, factory.ts:2653.

Why it matters (observed, not theoretical)

@agent-relay/sdk@10.6.2, dist/messaging/relaycast.js:540-544:

const capable  = nodes.filter((node) => this.nodeHasCapability(node, input.capability));
const live     = capable.filter((node) => node.live);
const eligible = live.filter((node) => this.nodeMapsRepo(node, input.repo));
if (eligible[0]) return { node: eligible[0] };

nodeMapsRepo returns true unconditionally when repo is undefined, so every live capable node stays eligible and eligible[0] wins.

Against my live workspace, dispatching AgentWorkforce/relayed:

repo passed eligible placement picks
undefined (what factory sends today) [hoopsheet, sf-mini] hoopsheet — my laptop
AgentWorkforce/relayed [sf-mini] sf-mini — the correct remote node

sf-mini advertises repoKeys=["AgentWorkforce/relayed"] (derived from factory.node.json clonePaths); the laptop node advertises none. The repo tags the node registration is careful to publish are never consulted on the dispatch path.

With the one-line fix applied, factory dispatch ... --backend relay placed ar-47-impl-relayed on sf-mini — verified independently of the PR: the spawn ack reported node = sf-mini, agent-relay node agent list on sf-mini listed the agent, a new codex PID appeared there, and the branch agent/47-cross-node-proof was created in sf-mini's clone. Zero ar-47 processes existed on the orchestrator.

Proposed fix

node: spec.node ?? 'self',
repo: spec.repo,

at factory.ts:2484 and :2653.

Secondary issues found alongside

  1. eligible[0] is not "least-loaded". relay-fleet-client.ts comments that the engine "picks the least-loaded eligible node" (per feat(fleet): dispatch the relay backend through @agent-relay/sdk placement #71), but the installed SDK selects the first match client-side. In my test, sf-mini (0 agents) lost to hoopsheet (2 active agents). Either the comment or the selection is wrong.
  2. A missing eligible node hangs for one hour, silently. createFleet never sets placementTtlMs, so ttlMs defaults to 60 * 60 * 1000 and the spawn queues for an hour before throwing placement_ttl_expired. Compounding it, src/cli/fleet.ts builds the relay client without a logger, so this.#log is a no-op sink and even the SDK's Placement queued: ... lines are swallowed. A misconfigured fleet looks like a hang, not an error. Consider a short default TTL (or failFast) for one-shot dispatch, and wiring the logger.
  3. The implementer's node is not targetable. routeImplementerSpec hardcodes node: 'self' (the reviewer honors reviewer.node). With no --node/config override, repo tags are the only way to steer implement work — which is exactly what issue [factory] p7: scope single|workflow|team driven by issue labels #1 above disables.

Refs: AgentWorkforce/cloud#2655, AgentWorkforce/cloud#2656, #71, #73.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions