What can a team of specialised AI agents accomplish that a single agent cannot, and where are businesses seeing a practical benefit?
“A team of specialised AIs can decompose an incredibly complicated task into smaller, bite-sized pieces, with each specialised agent taking ownership of the piece they are best suited for. This has a number of benefits. Most importantly, having a team of agents take on bite-sized tasks allows you to better manage the context window for each agent, because overflowing a context window is where you get a lot of problems with hallucinations and drift.”
How should an organisation divide work between agents? What makes a handover between them reliable?
“Organisations would do well to divide agents between tasks they’re well-suited for and access they already have. Not every agent should have access to every surface in the organisation, so strong access controls and provisioning specific agents for specific access are key security components. That also makes it more obvious which work should go where.”
When agents plan or negotiate with one another, how can businesses tell whether they are making progress towards the intended outcome or simply reinforcing each other’s mistakes?
“A key to using agents well is to build in verifiable outcomes of the agent’s work and key milestones for the task, so you have specific numbers or objectives to track against. This is similar to the way the CEO of a large company doesn’t know what every individual contributor is doing, but knows the metrics and can see if numbers are tracking in the same way. It’s also helpful to have a separate agent whose entire job is to adversarially review the output and work of the other agents on the team.”
How can an error made by one agent spread through a chain of agents, and what controls are most effective at stopping it?
“A key insight many AI labs have shared is to have a primary orchestrator agent and an adversarially reviewing agent (i.e., an agent that reviews the output of other agents). Also, before you begin working on any project, you should have an incredibly well-defined spec doc with specific tests written beforehand that the orchestrator and review agent can use to measure the other agents’ output.”
What should be recorded at each handover so that a person can reconstruct why the agents reached a decision or took an action?
“In technical terms, I would say that someone monitoring agents in a very secure, high-risk, code-heavy environment should use telemetry tools,like Langfuse, to trace agents’ actual reasoning, tool calls, and outputs in sensitive flows. For a lot of business operations, that might be overkill, but a helpful trick is asking an agent to create an HTML document outlining exactly the decision-making process and architecture of its final output in an aesthetically pleasing way (for the human reviewer to be able to quickly understand it at a glance).”
If two agents disagree or produce conflicting recommendations, who or what should resolve the dispute?
“There should be a primary orchestration agent that can settle these disputes before they have to be escalated to a human reviewer. The orchestrator agent should be powered by a large, preferably frontier, model, whereas the smaller agents may use a smaller / lighter model, maybe not even a reasoning model.”
At what point does adding more agents make a workflow harder to manage than the process it was meant to improve? How would you recognise that point?
“Generally, the right shape of the problem for agentic workflows is verifiable outcomes that a human or agent can easily check to determine whether the task was completed correctly. If you’re using an agentic workflow on a problem that doesn’t have verifiable outcomes, if you end up spending a few weeks just turning imaginary knobs and trying to tweak it into a way that fits your purpose, you’re probably going to end up down a long, windy road that won’t end, so it’s probably best to think about a new way to approach the problem.
“Agents are helpful across a number of different domains, but some problems are better suited for them than others. If you find yourself spending days “turning knobs,” it might be worth your time to explore how they could be used to help out other workflows, or at least change the shape of the work that you need to be doing in a different task.”
Dane Sherrets, Staff Innovations Architect at HackerOne.
Dane is an experienced Staff Innovations Architect at HackerOne, helping large enterprises and governments leverage bug bounty programs and AI red teaming services to minimise growing cyber risks. Before joining HackerOne, Dane worked as a mobile application security specialist, helping organisations use tools to uncover vulnerabilities in their mobile apps. Dane is passionate about leveraging advancements in cryptography to preserve privacy on the internet. In his free time, Dane enjoys bug bounty hunting and tinkering with EVM-based smart contracts.