fix(agent-teams): keep long-running teams recoverable (#1426)

Long Agent Teams runs lost members for good: a truncated provider stream
ended a member's turn with nobody to wake it, the desktop Stop button and
every lead restart killed all members and marked the plan interrupted,
mail sent to a stopped member landed in an inbox nothing read, and a lead
kept inside one long turn never saw member reports. Aligned with the
official CLI 2.1.284 and verified with DeepSeek Flash through a
fault-injecting proxy.

Stream recovery
- Re-send a stream that breaks before any tool ran (proxy truncation,
  transport errors), with the existing retry budget and backoff; the
  desktop drops the discarded attempt's tool cards and todo update.

Desktop team runtime (teamPlanRuntime)
- The server supervises members: a stopped member restarts from its own
  transcript when messaged; transient failures continue automatically
  (15s/45s/2m/5m/10m) and only exhausted retries reach the lead; ready
  dependent tasks wake their owner; a crash-loop guard ignores user stops.
- Stop pauses the team instead of ending it; the lead's next user message
  is followed by a notice listing the stopped members and their open
  tasks. Lead restarts (model/permission switch, crash) keep members;
  server restarts re-own the team. Teams end on /clear or session delete.
- Approving a plan no longer races a concurrent plan read into
  "Launch ownership was lost".

Mailbox and messaging
- Atomic inbox writes, identity-based read marking, read history files,
  idle notifications with result/failureReason, and write failures
  reported instead of "Message sent". External builds keep the official
  between-turn delivery to the lead.
- SendMessage resumes non-running in-process teammates from their
  transcript, notes restarting desktop members, queues mail for members
  of a plan awaiting approval, and rejects unknown names.

CLI in-process teammates
- Compaction uses the teammate's own controller and real history and no
  longer kills it on error; failed turns are classified and continued;
  the turn-end mailbox drains as one batch; one durable transcript per
  teammate.

Lead behaviour
- An unmet /goal ends the lead turn while members work, so member reports
  arrive; WaitSessions on own team members returns immediately.

Desktop UI
- Member states for stopped, auto-retrying and failed, with reason,
  countdown and recovery hint in all five locales.

Tests and tooling
- Regression tests for every behaviour above; module mocks in four test
  files are restored after use so the single-process coverage run is not
  polluted; the desktop smoke asserts the new Stop semantics.
This commit is contained in:
程序员阿江-Relakkes
2026-10-03 17:07:30 +08:00
committed by GitHub
parent 6d8071be8e
commit fd42b0ca05
104 changed files with 9896 additions and 1070 deletions
@@ -109,10 +109,15 @@ export async function runDesktopUiTeamPlanSmoke(options: {
await browserStep(['wait', '--fn', '!document.querySelector("[data-testid=team-plan-open]") && !document.querySelector("[data-testid=team-plan-approve]")'])
await browserStep(['wait', 'button[aria-label="Stop"]'])
await browserStep(['click', 'button[aria-label="Stop"]'])
await until(async () => (await getPlan()).state === 'interrupted', 'visible Stop to interrupt the running team')
// Stop pauses an approved team: every worker stops and keeps its saved
// conversation, and the plan stays running so a later message resumes it.
await until(async () => {
const team = JSON.parse(readFileSync(join(configDir, 'teams', TEAM_SMOKE_TEAM, 'config.json'), 'utf8'))
return team.members.filter((member: { name: string }) => member.name !== 'team-lead').every((member: { isActive?: boolean }) => member.isActive === false)
}, 'stopped workers to become inactive')
return team.members
.filter((member: { name: string }) => member.name !== 'team-lead')
.every((member: { isActive?: boolean; terminated?: boolean }) => member.isActive === false && member.terminated === true)
}, 'visible Stop to stop every running team worker')
const paused = await getPlan()
if (paused.state !== 'running') throw new Error(`Stop ended the approved team instead of pausing it (plan ${paused.state})`)
await browserStep(['screenshot', join(artifactDir, 'team-review-stopped.png')], { allowFailure: true })
}
@@ -12,6 +12,14 @@ const checks: Check[] = [
title: 'Agent Teams plan sidecar compatibility and approval recovery',
command: ['bun', 'test', './src/utils/swarm/teamPlanStore.test.ts', './src/server/services/teamPlanService.test.ts'],
},
{
title: 'Agent Teams legacy mailbox migration to unread inbox plus history',
command: ['bun', 'test', './src/utils/teammateMailbox.test.ts', '--test-name-pattern', 'legacy inbox files'],
},
{
title: 'Agent Teams teammate resume from agent metadata written before the teammate fields',
command: ['bun', 'test', './src/utils/swarm/inProcessRunner.resume.test.ts', '--test-name-pattern', 'metadata written before'],
},
{
title: 'Session collaboration state migration and recovery',
command: ['bun', 'test', './src/server/services/sessionCollaborationService.test.ts', '--test-name-pattern', 'migrat|recover'],