Design Agent Teams Around the Work

A planner, implementer, and reviewer give agent work a useful backbone. Planning makes the outcome explicit, implementation gives one actor responsibility for delivery, and review creates a checkpoint before the result is accepted. The sequence turns separate model calls into a process a company can direct and inspect.

That backbone becomes more useful when its execution can adapt to the work. A factual lookup may need one capable agent and a checkable source. A difficult product decision can benefit from independent proposals before a final choice. A code repair can use executable tests as immediate feedback, followed by review of the requirements those tests cannot cover. Each task still has clear ownership while the collaboration pattern changes to fit its dependencies and risk.

A recent paper, Toward an Organizational Science of Multi-Agent LLM Systems, provides a framework for making those choices deliberately. Huan Chen and his co-authors separate three parts of an agent organization: who participates, how they coordinate, and which collaboration algorithm combines their work. Varying those parts independently makes the organization configurable, measurable, and easier to improve from experience.

Three decisions hide inside every agent team

Choosing an agent team involves several connected decisions. A role identifies who contributes, coordination defines how responsibility and information move between participants, and the collaboration protocol turns those contributions into a result. Separating these layers lets a company preserve clear ownership while selecting an execution pattern that fits the task. The paper describes them as participants, a coordination structure, and a collaboration protocol, or the who, the how, and the algorithm.

Decision	What it controls	Example
Who participates	Roles, models, tools, and responsibilities	Researcher, implementer, domain reviewer
How they coordinate	Authority, handoffs, shared information, and accountability	One lead assigns work, peers adjust together, or outputs follow a standard form
How work is combined	The algorithm that turns several contributions into one result	Voting, debate, Reflexion, mixture-of-agents, or plan-and-execute

These choices are easy to confuse because role names often imply a process. Calling one agent the planner and another the reviewer suggests that planning happens first and review happens last. The names still leave many operational questions unanswered.

Does the implementer have to follow the plan? Can it challenge an assumption before starting? Does the reviewer see the complete execution or only the final artifact? Can the reviewer reject the result, request a bounded correction, or merely leave advice? Who writes the final version after disagreement? Which tools and evidence can each participant access?

Two teams can have the same role labels and run very different organizations. One may use the planner's output as binding execution context and let the reviewer block completion. Another may put three personas in a group chat and ask them to reach consensus. The org chart looks similar while the information flow, authority, and result are different.

A role is real when it changes the work

The paper tests this with RACI, a responsibility-assignment framework used to clarify four kinds of participation. The Responsible participant performs the work, the Accountable participant owns the final outcome, Consulted participants provide input, and Informed participants are kept aware of the result. Moving the Accountable label had little effect in protocols that never read it. Voting and debate continued to behave the same way because the label changed no step in their execution.

The result changed when the protocol gave the label a consequence. In the paper's LLM-Blender implementation, the Accountable agent performs the final fusion. Moving accountability from the Chair to the Critic therefore changed which model wrote the deliverable. Across repeated HotpotQA runs, that change produced a consistent directional difference. The winning placement did not remain stable across datasets and model bindings, which is precisely why the authors argue against hard-coding it.

This gives agent builders a practical test for every role:

Role	A consequence that makes it operational
Planner	Its Work Spec controls the context, constraints, and acceptance criteria used in execution
Implementer	It receives the tools, budget, environment, and authority needed to change the artifact
Reviewer	It can approve, request revision, send the work back for replanning, or block completion
Team lead	It controls queue triage, assignment, deferral, and escalation
Verifier	Its result determines whether the workflow continues, retries, or asks a person
Accountable owner	It makes or authorizes the final decision and remains identifiable afterward

A different prompt can help an agent adopt a perspective. It does not by itself create organizational authority. Durable roles need to change at least one of the inputs, tools, decisions, state transitions, or outputs in the execution path.

This is where many simulated AI companies remain decorative. A screen can show a chief executive, a researcher, and a marketing director. If every model receives similar context, has similar access, and sends prose back to the same chat, the hierarchy has supplied characters rather than an organization.

Different work needs different collaboration

The paper implements six collaboration protocols behind a common interface: voting, mixture-of-agents, a ranking blender, debate, Reflexion, and plan-and-execute. Its broader contribution is the ability to hold the team constant while changing how that team works.

That separation matters because tasks have different dependency structures and different sources of truth.

Shape of work	Sensible starting pattern	Why it fits
Factual lookup with a checkable answer	One capable agent or several independent answers with voting	Extra coordination adds little when correctness is easy to compare
Open-ended strategic choice	Independent proposals followed by comparison or debate	Early agreement can hide a plausible alternative
Complex decomposable project	Planner, bounded parallel work, synthesis, and review	Explicit handoffs help manage dependencies
Code repair	Implementer, executable feedback, bounded repair, then review where tests are inconclusive	The test or reproduction can supply a stronger signal than another opinion
High-risk external action	Preparation, deterministic checks, named human authorization, execution	The decision needs authority and consequence awareness
Repeated stable operation	One trusted executor with verifiers and exception routing	Repeating full deliberation wastes time and money

These are starting hypotheses. A code task with weak tests may need independent review. A factual question may be ambiguous enough to warrant source comparison. A strategic decision can be simple when the company has already established the policy that governs it.

The smallest sufficient organization is usually the best place to begin. More agents add model calls, latency, duplicated context, and more opportunities for a weak handoff. They earn that cost when they contribute independent evidence, useful specialization, broader search, or a correction mechanism the task actually needs.

The evidence supports experimentation, not a universal recipe

The paper's experiments are useful because the limitations are visible.

In a controlled simulation of 400 mixed tasks, its Adaptive Org Router learned to select among the six protocols using simple task features. The reward combined estimated quality with a cost penalty based on relative model-call count. It reached 82% routing accuracy over its final 50 tasks and outperformed every fixed protocol in that synthetic environment.

The synthetic environment had a known answer for which protocol should fit each task type. Production work does not provide that oracle. Quality can arrive through tests, customer response, human review, later rollback, or an evaluator that carries its own biases.

The real-model studies were much smaller. Individual protocol comparisons used ten items per cell. The online router completed an 18-round benchmark stream and a separate 20-round stream that included an LLM judge. Those runs show that the machinery can consume real rewards, not that it has learned a dependable production policy.

The strongest solo model also matched or exceeded the multi-agent protocols on quality in the reported benchmark comparisons. Multi-agent work should therefore have to justify its coordination cost. Several models discussing an answer can look more rigorous while producing no better result than one well-chosen model.

The current router learns which protocol to use while holding the organization fixed. The paper explains how the action space could expand to combinations of organization, coordination, protocol, and per-role model bindings, but that larger problem remains future work. Long-running tool use, human approval loops, changing external state, and real company outcomes are outside the experiments.

The reliable conclusion is narrower and more useful: organization and protocol both affect results, their interaction can change with the task and model binding, and a fixed arrangement should be treated as a prior rather than a law.

Recording a routing decision is only the beginning

A team can start learning before it has an adaptive router. It first needs to preserve enough evidence to reconstruct what happened.

For each repeated class of work, record:

• the task type, risk, and relevant context
• the participants and the exact models they used
• the coordination and collaboration pattern
• who produced the final artifact
• the original routing recommendation and any human change
• verification, review, correction, and retry outcomes
• elapsed time and total cost
• later rejection, rollback, customer response, or other outcome evidence

A human reassignment is valuable evidence that the first choice did not fit the person's intent. It does not establish that the replacement performed better. That requires following the task through review and into whatever later evidence can judge the result.

The same caution applies to approval. Keeping an automatic recommendation is not clean evidence that rank one was optimal. The person may have agreed, lacked time to compare alternatives, or considered the difference too small to justify intervention.

This is why organizational learning belongs in structured execution history rather than prose memory alone. Memory can carry a useful lesson such as “use an independent source reviewer for pricing research.” The underlying tasks, versions, costs, and outcomes should remain available so the lesson can be challenged when the work or models change.

How Task Machine handles the three roles today

A Task Machine task can resolve a planner, an implementer, and a final reviewer. These are role slots, not a promise that three different actors participate every time. Defaults can resolve the same person or agent into more than one role, and a simple task can avoid unnecessary ceremony.

Each role changes execution. The planner writes the Work Spec that carries the outcome, constraints, risk assessment, and acceptance criteria into later work. The implementer is the active assignee with responsibility for execution. The final reviewer owns the fresh review that can approve the result, request revision, or send the approach back for reconsideration.

When an agent implementer submits work to a human final reviewer, Task Machine first runs a separate agent pre-review. A passing pre-review releases the decision to the human Inbox. A correction must pass through review again, and repository work binds review evidence to the exact pull-request revision that was examined.

Teams add another layer. A team can own a queue until its lead routes a task to a concrete implementer, while also selecting the planner and reviewer. The routing recommendation, rationale, and proposed values are recorded. When approval is required, a person can change the complete decision from the Inbox before it applies.

Task Machine also selects models independently for planning, implementation, review, verification, and follow-up when automatic model routing is used. Workflow graphs can express different sequences of agent work, checks, questions, approvals, branches, and retries.

Together, these pieces make roles, routing, and collaboration inspectable while preserving a clear owner for every decision. A team can keep a stable operating model and still vary its workflow, role assignments, and model choices as the work requires.

The routing record also creates a foundation for careful improvement. Comparing recommendations, human changes, verification, quality, cost, and later corrections can show which configurations serve recurring work well. Shadow recommendations can then test routing improvements against real outcomes before they influence live execution.

Audit the workflow before adding another agent

Take one recurring workflow and trace it from assignment to accepted result.

• Can each role be tied to a concrete change in context, tools, authority, or state?
• Could one role disappear without changing the result?
• Does the collaboration pattern match the dependencies in the work?
• Is the final artifact attributable to the actor and model that produced it?
• Can a human change to the routing be connected to later quality and cost?
• Are deterministic checks doing work that another agent would otherwise repeat?
• Would a simpler solo run provide a fair baseline?

An effective agent organization reflects the work it repeatedly performs. Planner, implementer, and reviewer provide a strong default for consequential tasks, while explicit execution contracts and outcome evidence help the company refine how those roles collaborate over time.

Continue with how execution evidence can become a governed organizational change.