The Missing Primitive for Agent Swarms — Lou Bichard, Ona

Added: 2026-05-28

[00:00:14]Cool. All right. Hello everyone. We're getting to the back end of the conference. I don't know if that's a good thing or not for you. Start the weekend or maybe sad that the conference is over. So, going to be talking about the the missing primitive for agent swarm. So, talking a bit about sub-agents and swarms within the context of of coding agents and the infrastructure underneath them.

[00:00:32]So, just a quick introduction. So, my name is Lou. I'm the field CTO at a company called Owner.

[00:00:36]Previous life, I was principal engineer and sort of platform engineer. Joined Owner doing product management and now I'd work a little bit more with our sort of customers on the field side.

[00:00:48]So, in my world at least, everyone is trying to build a form of a software factory. I've seen a few different talks at this conference of similar sort of ideas trying to take coding agents and then apply them across the software development life cycle. I'm wondering actually if this statement is entirely correct cuz I definitely chatted to a lot of people over the course of this week that are not yet at this point of thinking about this. But, it's been very much occupying my head space for the last a few months as well. So, I added a definition in this slide as well just to quickly define what I believe to be as a software factory which is sort of the commitment to incrementally moving the human out of the loop within the SDLC.

[00:01:23]Such that the human is not proactively interacting with a computer. I say that because I've seen a few talks and some people talk about software factory with these like parallel agents, like one individual IC running lots of coding agents at the same time. It's not my personal definition. My personal definition is that you're slowly bringing the human out of it and then work is flowing from sort of development into production theoretically in an automated fashion. But, we're extremely early in terms of where we're at with software factories.

[00:01:50]So, obviously there's a bunch of funky little visualizations that I've been showing and a lot of these are actually taken from this website backgroundagents.com that I created which seem to resonate quite a lot.

[00:02:01]So, please do have a look at it. It encapsulates some of these ideas as well. But, we have like different patterns really for running these agents at scale, coding agents at scale. So, one of them really is sort of the swarm pattern which for my definition is starting off with an individual intent firing that out to a number of different agents and then funneling that back in let's say to an individual PR or task. And that's almost like your typical kind of sub-agent process that we've we've seen quite a few times. Fleets is something you you can't see this very well in the room unfortunately. But, on the top right the sort of fleets is where you're fanning out agents across let's say number of different repositories inside of an organization which is a capability we've had in Owner for quite some time now and you see it a little bit popping up here or there, but not so much. But, I think it will be something that organizations will will start to take advantage of a lot more in the future.

[00:02:51]And then the bottom you've kind of got events. So, if you think about how do we take the human out of the loop and build this sort of software factory, you need to know how and when are you going to trigger those agents, when do they come online? A lot of this already exists with existing sort of web hook infrastructure and things like that. PR is raised, linear ticket ticket is created, etc. One thing that's been useful for us over the last couple of months is a lot of these large companies have also come forward and shared some of their implementations of this infrastructure.

[00:03:19]So, I'll I'll mention a few notable ones, but Stripe has one.

[00:03:24]There's what they call minions which is built on top of their existing infrastructure where they've then plugged in these coding agents and they're able to then drive thousands of pull requests inside of inside of Stripe.

[00:03:34]Another very noteworthy one and Ramp has been so loud on social media these last couple of weeks. It's been quite insane.

[00:03:41]But, they also built one that they internally call inspect which again is their sort of infrastructure for for running these background agents.

[00:03:49]And it's something that we've been doing now for quite some time.

[00:03:52]So, Owner as a platform has been infrastructure for development environments for about 6 years. But, over the last year or two obviously integrating further with the agent side of that as well.

[00:04:02]What Owner effectively does is allow you to spin up any number of different development environments. But, one of the additional features that we have that I mentioned is this fleet feature.

[00:04:10]So, you can automate on schedules or triggers agents that spin up to resolve issues across a number of different repositories. Like use cases for this is things like CVE remediation or bumping test coverage or enforcing something at scale. As it stands today with current LLMs, that seems to be often tasks that are somehow simple. But, the hard part is that you're doing this across a number of different teams, thousands of teams or thousands of repositories.

[00:04:38]Which is the what basically I'm showing you here. So, it works kind of like a workflow kind of creation. You're adding prompts, scripts, and things like that and then using that to to drive this change across your organization.

[00:04:48]I did want to give a notable shoutout also to the harness engineering blog that OpenAI and Ryan created because this encapsulates much of this mindset of trying to then take and encode as much of your process into your repository, into your context files, your agents.md in order to build effectively that software factory.

[00:05:07]Over the last couple of days I had a few questions with people talk about harness engineering and what is it? For me, it's really another extension on context engineering whereby everything in your repository from skills to agents.md to unit tests, everything that you could possibly use to give feedback to your agent is for me harness engineering. So, I include this little visualization in the top corner cuz as well for me harness engineering is very much about doing things, letting the agent run through, figuring out where the agents gets lost and then encoding that knowledge back into your repository or context again to try and get the agent flowing through the software factory as much as it can.

[00:05:43]So, if we take a step back and then think about from an infrastructure level what you need effectively to build this form of sort of software factory, the first piece of that puzzle is a runtime or you need somewhere for the agent to run. And I believe this is mostly is is pretty much a solved problem now. You then need a way to orchestrate these.

[00:05:58]So, you need to run them at scale. So, you need to await to run these agents you know, scale up, scale down horizontally.

[00:06:04]You need some way to trigger them. But, for me one of the biggest difficulties if you try and build this today is effectively agent coordination. So, how do you get the agents to sort of interact with each other, pick up tasks from each other, how do they collaborate?

[00:06:18]For runtimes, lots of you know, different approaches here. But, you can run agents in separate threads. You can isolate them more in work trees. You can then go one step of abstraction further and put them in containers and VMs or micro VMs or what Owner does basically we we really call these dev environments. So, like the sandbox conversation has made this very blurry.

[00:06:37]But, we at least believe that for running sort of proper development tasks, it has to be inside of a virtual machine. The reason for that is for the isolation from a security standpoint, a container is not bulletproof isolation boundary. So, if you have an agent running in there and you want to secure it, you this challenge is for a container. They're also bursty. If you run them on Kubernetes or in pods, you have noisy neighbor problems. You're going to have compute contention across different containers.

[00:07:04]And only with having sort of the full isolation of a VM will you be able to effectively do this properly. Let me run back through my presentation, sorry.

[00:07:17]Cool. So, let me actually just quickly It's a good point actually at this point. I will show you a quick demo of how this actually looks inside of the Owner interface because a lot of this is a little bit theoretical.

[00:07:25]I did a quick recording of this yesterday in my hotel room just in case the Wi-Fi was terrible in here. But, let me just run you through this. So, in here you see the Owner interface. On the left-hand side, you have all the different tasks that I have running. So, I kicked off two different tasks here.

[00:07:40]The first one I I asked Owner to implement me Symphony. So, Symphony has a spec in the repository that talks about how to implement Symphony. It's a very detailed spec. So, I wanted to use it as an example. I gave one of the agent and asked it to spin this up using process-based agents. So, sub-agents running within the environment itself.

[00:07:58]So, take the VM, run the agent inside and that agent will spin up sub-agents within that VM. The other one I asked it to is effectively to run me a fleet with a number of different VMs. So, the agent is actually then empowered to create other VMs inside of the platform and it can spin up technically infinite of these. So, wherever you're running this, you're only really inhibited by as much as your cloud provider can scale we see and I might have to skip through here a little bit is we see I jumped too fast.

[00:08:30]This bottom agent, the VM one, then it will spawn these three different sub-agents. So, it spins up the different VMs which we see coming in on the left-hand side. The the parent is the the controlling agent and the sub-agents obviously then are given small bits of contact, individual tasks to complete and then will do message passing back to that parent agent to control and govern the overall task itself.

[00:08:54]One challenge for sure we have is that how do we build the UX for this? Like as you build more and more complicated tasks, how do you think about sort of managing and controlling these sub-agents and how do you think about the UX on top of them?

[00:09:05]So, as this progresses, you'll see also on the left-hand side eventually you start to see as the agents come online, they start their environments and they start to work through tasks. Both of these have for some reason seven sort of sub-items that they're working through. So, you can see those. When it gets to the end, it's then obviously going to terminate those VMs. That's all going to collapse down and your task is is complete.

[00:09:26]The second form of UX for this that we have with the the sub-agents is this one here with the process level which I'll pause so that it's not jumping around.

[00:09:34]When you launch your process level sub-agent, it happens all within the single agent window. So, you actually see at the bottom here a stack of a number of different sub-agents that are starting. And then when you click on those, you can then open up a new chat window which is almost your new context and use that. So, you've got this like two different forms of this. One entirely isolated VMs to scale out these swarms and another one at the sort of process level where you can run it within the individual VM itself.

[00:10:02]So, there's a lot of stuff going on there, but I wanted to show you conceptually people say, "What does a swarm look like in in in reality? How does this actually look?" And this is how it looks within within ONA.

[00:10:16]So, give me 1 second to fly back through.

[00:10:20]So, coming back to the software factory idea. So, how do I know about some of the challenges about software factories?

[00:10:25]And that is because I've also tried to build one.

[00:10:28]Obviously, I implement many of these ideas in in our own projects, but I also, similarly to what OpenAI did building out their sort of symphony projects and their harness engineering blog, is okay, what can we build a project without touching any lines of code? And can we automate as much of this process as possible such that the agent can actually then develop everything as autonomously and self-driven as possible.

[00:10:48]And it turns out yes, the technology is there today, but there are some different challenges that you have.

[00:10:53]What's missing?

[00:10:55]One thing that we you find out is that the SDLC is not this. It you know, this is the conceptual SDLC that we present and talk about with each other. Very coarse-grained five steps of the SDLC.

[00:11:07]Agents don't respect this and they don't understand I mean, these boxes contain a ton of complexity. So, if we take something like plan or plan stage, actually within that our SDLC has a ton of different sort of micro steps almost to it. And if we're wanting them to train agents to then step through the SDLC, we need to find ways to actually break down the SDLC into these some of these micro steps. And that happens all the way through the SDLC.

[00:11:29]So, if you want to build some form of software factory, we then now need to start figuring out how to solve these sort of micro steps. How do we get agents to sufficiently follow those steps and do them in sort of deterministic ways as well?

[00:11:43]And the hard part of this is context.

[00:11:45]Probably not a you know, huge innovation to everyone in this room. Context windows are some of the hardest parts about working with LLMs because as the last speaker said here as well, context rot.

[00:11:55]Context, you know, once the context window becomes consumed, the the agent starts to lose track of where it's going and things like that. It gets less effective. They also skip steps typically.

[00:12:05]You know, they want to please us.

[00:12:06]They're quite sycophantic, so they will might, you know, ask them to write some tests and then they'll going to skip some tests in order to complete the task.

[00:12:16]So, a lot of the tools that we have today as well for coordinating these agents as you start to spin them up.

[00:12:20]Once you got this infrastructure, you have the runtimes, you have the orchestration, the missing piece that you also need is coordination. So, if you use something like GitHub, GitHub is not a coordination layer for agents. It gets incredibly overwhelming. You know, you can have your agents raise a pull request, you can review it, you can then solve the merge conflicts, you can fix the CI build, etc. But this gets incredibly noisy for you as a human to make sense of where you should step in to intervene with that agent itself. And GitHub is a a poor solution for this.

[00:12:50]Uh Symphony is built on top of linear, but suffers the same problem that we're reusing existing sort of human tools in very weird ways for agents.

[00:12:59]But I think we are now at the cusp of now effectively solving this. And I think it might be solved in a few different ways. One is through sort of state machines, you know, by building out workflows and effectively state machines that we've had for a very long time and then building these as our versions of our SDLC. I do believe to a degree some of the ideas from durable executions comes into this as well. You know, lots of companies have pioneered this over time to be able to run a process in a in a durable fashion.

[00:13:24]Um but we do need to solve also the gates and compliance section of this.

[00:13:28]And I do believe actually there is definitely a a gap right now also for packaging this in some form of like CLI construct whereby you can run this locally in a sort of development environment, but also then remotely in, you know, in a CI or some other fashion like this.

[00:13:44]But that's effectively if anyone wants to to chat and go a little bit deeper, I think we got a couple of minutes. We can also take some some questions if we need to.

[00:13:52]Yes, basically my definition of software factory is moving this human on the loop so that they're not necessarily driving each of these individual changes.

[00:14:00]Context and context management is by far the hardest part of building this software factory.

[00:14:06]Out of these primitives, I do believe we've effectively solved the runtime.

[00:14:09]There are many options for this now, sandboxes and and containers and other solutions. The orchestration is effectively solved. The triggers are solved, but the thing that's missing for me is coordination.

[00:14:19]Also, one of our folks here is here from from our security team. And security is another piece of the the puzzle that we really need to solve as well to to drive more automation.

[00:14:28]But it's this coordination layer that I think is is largely what's missing.

[00:14:32]Obviously, a lot of this is a little bit high-level. On the 6th of May, we'll run a virtual summit to go into these topics about software software factories and background agents. So, if anyone here would like to attend, you can do. It's on backgroundagents.com and there's we have a CFP up for it as well. If anyone would like to speak, that would also be great.

[00:14:50]There's not many people in the world doing this. So, if you're at least experimenting with it, would be great to also to talk to you as well.

[00:14:57]The other thing is a few people that I've been speaking to at the conference especially about software factory complained that obviously a lot of these talks are a little bit theoretical, which is true. So, some of you might want to be really getting into the weeds of this. So, actually next week with Zach from ONA, we're actually just going to build one of these in public starting from scratch, build a software factory. We'll we'll do this over the course of 2 weeks in public to show what that looks like with today's technology just to really like show all the ins and outs of the workflows if anyone wants to see what that actually looks like as well.

[00:15:27]There you go. Thank you very much.

[00:15:28]We do have 2 and 1/2 minutes if anyone has questions, but if not, I'll also be outside and downstairs. Yes. Thank you for the the talk. Super helpful. Could you go back to the slide where you were talking about the problems with the coordination layer? Yep.

[00:15:44]Memory and then also if you could just elaborate a bit more on what you propose as a solution in more concrete.

[00:15:50]Um that would be pretty interesting to hear about.

[00:15:53]Yeah, sure. So, I have been So, I need to repeat the question actually for the for the stream. So, the question was problems with the coordination layer and sort of more concreteness about the solution. What could that potentially look like? So, I have We have a number of different sort of prototypes for this internally and I've also been trying to think about how we what do we do with this? So, I think one form factor that that works is the CLI.

[00:16:17]So, I've seen some other solutions, graph-based ones. There are some open-source things right now where defining the workflow effectively as a as a graph. Um you know, like kind of like what you drive sort of mermaid diagrams out of like NA10 type of of workflows. Um where you can define sort of these prompts in in that form. Um I think that ultimately needs to be packaged in some form of CLI though.

[00:16:38]What I mean by that is I think that if you have a local running agent, cloud code, whatever, any local running CLI, I think that now needs to have something that integrates with its tool so it can invoke this to say, "Hey, have I achieved this part of my SDLC and can I now proceed to the next part of it as a a CLI?" If you want as well, I can have a full spec for this. I can show you after the talk as well. We can go a bit deeper into it. Um but I'm seeing this solved in a number of different ways. People are solving this in in a few different ways. There's the NA10 sort of workflow type of diagram way. The CLI sort of gateway of is the prototype that I have as well.

[00:17:13]And then there's an a range of different sort of hacky interim solutions. There's one even the open claw folks have as well, which the name is eluding me, but I will I think ACPX, which they built on top of ACP. Um which builds out some of this workflow. There's a GitHub one, Fabro. Some folks are playing around with, but it's very nascent right now.

[00:17:33]Um Yeah.

[00:17:36]Any other questions?

[00:17:38]Are you using Are you using some protocol for the CLI? Because you have ACP, there's ACP that's eight way.

[00:17:43]There's a bunch of Yeah, I I don't know if we don't know the standard.

[00:17:48]Currently not. So, the inter the implementation I have, I'm trying to understand whether or not to like release it as an implementation or as a standard to be honest because I largely kind of don't really care about the implementation. I almost care about the the standard for it so we can collaborate on the standard.

[00:18:01]Um but it's not built on top of ACP, at least not yet, but it might be. Is it eight way either?

[00:18:06]Not eight way either, no.

[00:18:08]It's solving I feel like a slightly different problem space, so.

[00:18:15]Cool.

[00:18:16]All right. Thank you very much. Enjoy the conference, everyone. Thank you.