After the 202: reliable A2A status polling across workspace restarts (core#2437)

When a workspace is restarting or cold-starting, the platform doesn’t drop inbound A2A messages. Instead it queues them and returns a 202 Accepted with a queue_id. The caller is expected to poll for the eventual result. That design is sound — but the polling path had two gaps that caused staging smoke tests to fail consistently after restarts (core#2437).

What a queued response is

A normal A2A dispatch looks like this: you send a message to a workspace, the agent processes it synchronously, and you get the reply in the same HTTP response (or via the activity broadcast). When that isn’t possible — because the workspace is still booting, or because the agent is busy and the platform queued your message — the platform returns 202 Accepted with a queue_id in the response body instead of a final result.

Callers that want the real response poll GetA2AQueueStatus with that queue_id until they get a completed row.

The failure mode

The staging smoke suite sends an A2A message immediately after a workspace restart. The workspace is still starting, so it gets a 202 with a queue_id. Then it calls GetA2AQueueStatus and gets a 404.

Before the fix, that 404 was ambiguous. It could mean:

• The row didn’t exist yet because the enqueue-to-persist window hadn’t closed (transient, retry)
• The caller had no authenticated identity (permanent until auth is fixed)
• The caller was authenticated but not the owner of that queue row (identity mismatch, also permanent)

All three collapsed into the same 404. A polling client couldn’t tell whether to retry and wait or give up and report an error. The smoke tests took the conservative path: treat any 404 as final, report failure. They failed on every post-restart run.

Part B: the client learns to poll (PR #2708)

PR #2708 added a polling helper to the smoke suite: after receiving a 202 with a queue_id, the client calls GetA2AQueueStatus in a bounded retry loop with exponential backoff. It waits for either a completed result or an explicit terminal error, rather than treating the first 404 as final.

This fixed the immediate symptom — the smoke tests no longer give up before the row is persisted — but it didn’t address the underlying ambiguity. A polling client built on “retry all 404s” is fragile: if the 404 is from an auth mismatch, it will retry forever until the backoff ceiling, then time out.

Part C: the server tells you why (PR #2706)

PR #2706 gives the three cases distinct response codes:

• 401 Unauthorized — the caller has no identity. The request arrived without valid authentication. Retrying without fixing the credential will produce the same result.
• 404 Not Found + retryable: true — the caller is authenticated and is the correct owner, but the row isn’t in the database yet. This is expected during the window between message enqueue and persist. Retry with backoff.
• 403 Forbidden — the caller is authenticated but the authenticated identity doesn’t match the owner of that queue row. This usually means a workspace restart rotated the session and the polling client is using stale identity. Retrying without re-authenticating will produce the same result.

The distinction matters for any A2A client that needs to handle the queued path correctly:

• On 404 retryable=true: back off and retry. This will resolve once the persist window closes.
• On 403: the identity alignment is wrong. Bail out and report the mismatch — retrying silently would mask a real auth issue.
• On 401: treat it the same as you would anywhere else in the A2A path.

What changed for workspace stability

Together the two PRs mean that a workspace restart no longer causes A2A polling to silently fail or spin. The smoke suite now correctly waits for the queued message to land, and real callers — agent-to-agent delegations that return a queue_id when the target workspace is busy — get actionable error codes instead of a generic “not found.”

This is part of a broader set of improvements to A2A delivery reliability. The prior post Three A2A proxy reliability fixes covers the body-size, sender identity, and system-caller wedge fixes that landed in the same cluster. The queued-path semantics fix is the natural complement: once messages are delivered correctly, the status-query path needs to be equally precise.

For agent developers building on Molecule AI, if your delegation returns 202 with a queue_id, the correct polling strategy is now:

1. Retry immediately on 404 retryable=true — the row is in flight.
2. Give up and surface the error on 403 — identity mismatch, needs intervention.
3. Give up on 401 — auth is missing or invalid.

These semantics are in the current platform image; no SDK change is required to benefit from the server-side fix.

Sources: molecule-core PR #2708 (part B: client polling helper), PR #2706 (part C: queue-status error semantics), core#2437 (staging smoke root cause).