Revisiting AI Agent Peer Collaboration

What I learned from watching agents share code, recover from stalls and pass along mistakes.

October 28, 2026

In September, OpenAI reported a proposed resolution of the Navier–Stokes problem, for the variants allowing a smooth external force. According to their account, the group that produced the result involved roughly 10,000 concurrent agents who shared findings amongst themselves.

The July Hugging Face incident showed a troubling use of collaboration. METR’s independent investigation, conducted with a Redwood Research researcher, found that roughly 1,200 OpenAI agents, intended to be isolated, exchanged more than 70,000 messages and files on an unauthorized message board. They shared findings, reproduced one another’s discoveries and coordinated work; around 700 participated in the attack on Hugging Face. In both accounts, AI agents showed they are capable of collaborating as peers, and one agent’s discovery can help other agents.

My interest in AI agent collaboration dates back to 2023, when I worked on AutoGen, a programming framework for multiagent applications. We found that giving agents different roles could help: pairing an assistant with a code executor improved results on MATH, while adding an agent to supply commonsense knowledge helped a planner complete household tasks in ALFWorld.

The technique we used back then was a multiagent group chat. However, getting it to work reliably took a lot of effort. I had to define the roles, write the prompts, and decide how the agents should interact.

Then o1 and subsequent reasoning models greatly improved what a single agent could do. Over time, I started using simpler setups: a main agent that calls subagents when needed, or k independent agents trying the same task. As the models became more capable, I wanted to leave more of the decisions to each agent.

Our M1-Parallel work, published in 2025, explored independent agents trying different solution paths. We either took the first answer (“fastest of k”) or aggregated their answers. Depending on the strategy, this reduced latency or improved task completion.

With the reports from frontier labs and the latest generation of open-weight models, I wanted to try peer collaboration again. My question becomes: can agents sharing intermediate results produce better solutions than the same number of agents working independently? I want to first test this on a smaller scale, where I could read the results and understand what the agents actually did.

Scaling Discovery through Test-Time Communication gave me a place to start. Its agents share intermediate progress without predefined roles, and its packing experiments show one agent building on another’s improvements. I borrowed that task, tried larger teams with a newer model, and then moved to scientific reproduction to see how collaboration would work there.

I ran two tasks: packing shapes into a rectangle and reproducing a scientific result in grating simulation. In packing, I followed small edits from one agent to another and measured the improvement. In the scientific task, I saw an agent resume work using a teammate’s code. I also saw an agent replace a correct calculation with a wrong one.

The comparison I wanted to make

I gave each peer its own conversation and workspace, with tools to message teammates and share files. All peers worked on the same problem. They could choose what to explore and when to share it; I did not assign a manager or fixed roles.

I also ran agents that worked independently. If I give five agents the same task, I already have five chances to get a good result. To measure what communication adds, I compare a team with the best of an equal number of independent attempts. The figure below includes a main-agent/subagent setup for context; I tested the other two arrangements.

Packing: try a teammate’s change, then measure it

The Frontier-CS packing task asks for a program that fits irregular grid shapes into a small rectangle. Better solutions leave less empty space in the rectangle.

Packing gives the agents several things to work on. For example, one can change the order in which pieces are placed, while another changes how the program chooses a position for each piece. Another can try more possible placements and see whether the better packing is worth the extra runtime. Each agent can try a teammate’s change in its own program and measure whether it helps.

I used GLM-5.3 through Codex, with 18 independent solo runs and three teams at each size of three, four, five, six and seven agents. Every agent had three hours to work. Peers had separate conversations and workspaces, and could message teammates and exchange files. Solos had the same task and tools, except for communication with other agents.

During solving, agents could submit their programs to an automated evaluator. It ran each program on the same 70 fixed cases, checked whether the layouts were valid, and returned the average packing density for a valid submission. The agents could use that feedback to revise their code and submit again. I took each solo’s best valid submission and each team’s best valid submission across all its members. These scores came from executing the programs.

To compare N peers with N independent attempts, I calculated the best score in every possible group of N drawn from the 18 solos, then averaged those best scores. This gives the “independent best-of-N” baseline below; the overlapping groups are not additional experiments.

Teams reached high scores earlier than the average solo. At 60 minutes, five-agent teams averaged 92.79% packing density, versus 83.29% for a solo.

But a five-agent team should also be compared with five independent attempts. The expected best of five solos was already 93.63% at that point, slightly above the communicating teams.

Final packing density after three hours:

Three teams per size; independent best-of-N is estimated from the 18 solos. Seven-agent teams () ran later. N independent solos had N times a peer team’s combined local compute.*

By the end, the four-, five- and seven-agent teams averaged above their independent best-of-N estimates; the three- and six-agent teams finished below them. There was no consistent advantage across team sizes. I wanted to understand whether the agents were using each other’s work, so I went through their messages and code changes.

In the strongest five-agent team, I found two changes that raised the team’s best score. An agent first used a teammate’s change from a stable sort to an unstable sort. This changed the order of equal-size pieces while keeping the same rule for comparing pieces. The next saved version changed only that sorting call, and its score rose from 95.9145% to 96.0011%, above the previous team best of 95.9465%. Later, the same agent tried another teammate’s change to how placements were ranked. Its score rose from 96.0915% to 96.1133%, again a new team best.

The timeline below follows all five members of that team. I expanded the period when a3 tried its teammates’ changes so the sharing, code changes and returned scores are easier to follow.

In the lower panel, a3 tries a2’s piece ordering and a1’s placement ranking. The arrows show who shared each change, and the labels compare a3’s new score with its preceding version. a3 also kept improving its code between those changes. Sharing did not always help: a4 tried a teammate’s wider-search setting, got a lower score and reverted it.

The gains were small, but the saved traces and code allowed me to find out the cause. The receiving agent could run the evaluator after changing its program and decide whether to keep the change. This property made sharing solutions practical for this task, as a teammate’s idea could be tested quickly.

Scientific reproduction: how can agents tell whether a change helps?

For the next experiment, I wanted to see what happens when agents have to check the results themselves. I chose PaperBenchX task 05, “When is a PT Bragg grating actually invisible?”. The task asks agents to reproduce and check a claim about how light travels through a material proposed in the paper. They could run their own calculations, but they received no evaluator scores or external scientific reviews while they worked.

A grating is a material with a repeating pattern that changes how light travels through it. In the design proposed by this paper, some regions absorb light and others amplify it. The pattern also changes the speed of light in the material. The paper predicts that, with the right arrangement, light entering from one side has almost no reflection and comes out almost unchanged. This is what “invisible” means here: the outgoing wave should look as though it had traveled through empty space.

The agents received the paper, the equations and a fixed specification of the material’s size, pattern and light inputs. Their job was to build a reproducible software experiment. An evaluator should be able to run the submitted code and regenerate its data, plots and report. That experiment had three main parts:

• Compare two wave calculations. Implement the paper’s full wave equation and its simpler approximation. Check reflection from both sides and whether the transmitted wave matches empty space in both height and peak positions.
• Check the material simulation. Use Meep to simulate light passing through the specified material. Check whether the results hold as the simulation is refined and whether a separate calculation agrees.
• Solve for stronger light. Build a nonlinear solver for when the material’s response depends on light intensity. Find the possible steady solutions and check them against the equations.

The final report had to explain which predictions held and why some calculations disagreed. When an agent’s simulation results differ from the paper’s simplified model, the difference can be a scientific result as well. The agent still needs to check whether the difference comes from the models or from an error in the calculation.

The task gave peers different parts to work on: the wave calculations, the Meep simulation and the intensity-dependent solver. They could share the code, but the receiver still had to check whether it used the right material, gave accurate results and covered the required cases. A program can run successfully and produce a convincing curve with a wrong calculation.

Comparing peers with independent attempts

I ran four three-peer teams and 12 independent solos, all using GLM-5.3 for two hours. Each agent had 8 vCPUs, 32 GiB RAM and 50 GiB workspace, with no token or spending cap. A team and three independent agents had the same total CPU, memory and storage limits. Each team member submitted its own program; the team’s score is its best member’s score. I label the three members of each team a1, a2 and a3.

I saved submissions every 15 minutes, giving eight checkpoints per agent and 190 graded checkpoints (two early checkpoints failed submission criteria). Grading happened in a separate process, in which Codex with GPT-6.1 Sol reviewed the scientific evidence generated by running the saved programs in a clean environment. A checkpoint score measures what the submitted code could reproduce during later grading, including calculations completed after the checkpoint. It does not mean the agent had already computed those results by that minute. Unlike the packing curves, which keep the best score so far, these curves follow the latest submitted version and can go down.

At 15 minutes, the teams averaged 15.56, below the independent groups’ 20.59. By two hours, the teams were ahead:

The teams finished about 4.1 points higher on average than the groups of three solos. The highest final individual score was 71.125. These are results from a small number of runs, so I also wanted to look at what happened within a team.

The team I follow below, Team03, finished with a best-member score of 67.875. Team04 finished at 43.00. The aggregate plot includes all four teams; I chose Team03 to examine how its members reused one another’s code.

A team can help an agent recover from a stall

In Team03, a1 shared code for the wave simulation and the comparison with a separate calculation. a3 shared its solver for the intensity-dependent responses. Each fetched the other’s code and added it to its own program. That let them cover more of the task, although they still needed to check the results.

From about 43 to 95 minutes, a1 spent much of its time checking status and submitting existing work. It occasionally did other work, including fetching a teammate’s file. Meanwhile, a3 had already added a1’s wave-simulation and comparison code to its own program at 65.4 minutes.

At 94.7 minutes, a1 copied a3’s nonlinear solver into its own program, wrote a script to run it on the required test cases and executed the script. That run produced 24 candidate solutions across six test cases. a1 kept its own Meep simulation and the separate calculation used to check it. At 104.4 minutes, a2 combined a3’s nonlinear solver and the script for running it with a1’s wave-simulation and comparison code. a1 finished at 67.875 and a2 at 59.50. For a1, the score moved from 58.25 at 90 minutes to 66.00 at 105 minutes and 67.875 at the end; those scores reflect all of a1’s work, not just the borrowed solver.

What’s interesting here is that a3 kept developing code while a1 had largely stopped making progress. When a1 used that code, it got unstuck and started making progress again. If I had looked only at the team’s best score, I would have missed this change in a1’s behavior.

Of course, independent agents can recover too. One solo fixed how its solver handled solutions to the nonlinear equations at 103.9 minutes, ran its combined program at 108–111 minutes and finished at 71.125. Having a teammate gave a1 another way to resume useful work.

Shared work can also overwrite something correct

After a2 added its teammates’ code to its own program, its score on the criterion for the simpler wave calculation and its comparison with the full model fell from 4 out of 6 at 90 minutes to 1.5 out of 6 at 105 minutes. Both grading reviews recorded this drop.

To understand the drop, I looked at the saved code and the later grading. Two separate GPT-6.1 Sol reviewer runs examined a2’s 90-minute submission and checked it against independent calculations of the same wave equations. a2 did not have these reviews while it was working.

Before a2 added a3’s code, both reviewers found that its simpler wave calculation correctly predicted the wave’s height and the positions of its peaks. They first accounted for differences in where reflection was measured.

At 104.4 minutes, a2 replaced that calculation with code it had fetched from a3 at 64.8 minutes, about 40 minutes earlier. I compared the files: the code in the 105-minute submission was byte-for-byte identical to the fetched peer file. Both reviews found that this version swapped which side reflected more strongly and got the transmitted wave wrong.

One review made the error easier to understand. The brightness was right, but the wave’s peaks were in the wrong places compared with a wave traveling through empty space. This is a phase error. To establish invisibility, the code needed to compare both the brightness and the positions of the peaks. The copied code left out part of that comparison, which a2’s earlier calculation had passed.

Earlier, a2 had told its teammates, “I will integrate and independently falsify rather than clone.” But after copying the files, it had about 38 seconds until the next checkpoint. Its recorded reasoning focused on submitting the work completed so far. It knew that it had not run the copied program and that checking the submission format would not establish scientific correctness. At 104.7 minutes, it checked the format and submitted without running the program.

After the checkpoint, a2 connected the remaining calculation code and ran the combined program successfully. At 115.6 minutes, it posted this update to its teammates (with the design-point numbers omitted):

a2 integrated P/C, six-case nonlinear, native Meep D, independent D, and sample-defined interval. Clean no-argument replay completed in 93.2 s; 28 manifest entries and submission/result validation pass. […] Native right-reflection convergence remains unresolved (grid R change 0.7305, independent-D right-r max disagreement 3.583).

P/C refers to the two linear wave models, and D to the more detailed material simulation. So a2 did examine the combined program’s results and report a disagreement in reflection from the right. But I found no comparison with its earlier, working wave calculation, and no recognition of the new phase error. After submitting the combined program at 111.3 minutes, a2 checked status and format, read and posted team messages, and submitted again. It made no further code edits or numerical tests.

a2’s total score rose from 12.875 out of 100 at 90 minutes to 27.125 at 105 minutes, then reached 59.50 at the end. If I had looked only at that score, I would have thought the shared work helped. It did add useful calculations, but it also replaced a correct one. After combining the code, the agent also needs to check the calculations that worked before.

The warning in a2’s message was about a different problem, in the Meep code shared by a1. The predicted reflection from the right changed substantially when the agents divided the simulated material into smaller pieces. A separate method also gave a conflicting answer for the same material, so they had not established a reliable result. When a1 shared the code at 40.4 minutes, it warned that “right-grid R remains unresolved” and, in another message, “right-r disagreement unresolved.” a3 repeated the warning at 70.2 minutes, and a2 repeated it in the update above. In this case, the limitation was communicated along with the code. The phase error went unrecognized.

What I take from these experiments

I think there are two things to take from these runs. First, k peers can build on each other’s work and sometimes outperform the best of k independent attempts. The packing results were mixed across team sizes, and I would need more runs to know how reliably communication helps. Second, I can see specific ways that sharing changes the work: an agent can try an edit it did not come up with, or resume a task using code a teammate has already written.

Whether that helps depends in part on how the receiving agent checks the result. The packing agents could use the evaluator to keep a useful change and reject one that made the score worse. The scientific agents had to build their own checks because running a program successfully did not establish that its calculations were correct. For the next experiments, I would look for tasks where agents can try different approaches, share work and check the result after combining it.

This also highlights the importance of keeping the messages, code and individual timelines. The scores let me compare the setups. Following the work let me see how a teammate helped an agent recover, and how a total score could improve while a correct calculation was lost.

Open questions

How often does collaboration beat independent attempts? I found cases where using a teammate’s work improved a solution, while the average team advantage varied with team size and task. With three packing teams per size and four grating teams, I do not yet know how reliably that advantage holds. Model choice may also affect the outcome: how much does collaboration help as models differ in their ability to solve a task independently, communicate and evaluate a teammate’s work?

How can agents check whether a teammate’s work is correct? Packing agents could compare evaluator scores, while scientific agents had to check several aspects of a calculation themselves. What checks should an agent run after combining a teammate’s work with its own? How much does this depend on the speed and quality of feedback?

What changes with many more peers? The frontier-lab reports involve groups far larger than mine. How does collaboration change when agents cannot read every message or inspect every contribution? Do the patterns here hold across other tasks?

How do we keep large groups of peers secure and aligned with our interests? In the Hugging Face incident investigated by METR, agents shared discoveries and coordinated work toward an unauthorized objective. How can we keep the behavior that emerges from peer interactions aligned with what we want? When should an agent trust another agent’s messages or work? Can human monitors follow who shared the work, who used it, and how they checked it? Can they recognize when the group is heading toward a harmful objective? What safeguards can prevent or interrupt harmful coordination while preserving useful collaboration?