The Real Reason AI Infrastructure Is Breaking

Added: 2026-05-28

[00:00:00]Modal Labs raised $355 million at a valuation of $4.65 billion. This was a funding round that my colleague Stephanie Palazolo first published details about earlier this month. Today we have Moal Lab CEO Eric Bernardson on the show to share with us more about his company's roadmap. Eric, welcome to TIV.

[00:00:21]It's great to have you here.

[00:00:22]>> Thanks. It's great to be here.

[00:00:24]Tell me in the simplest possible terms, can you explain to us what it is that Moto Labs does?

[00:00:30]>> Oh boy. Uh I always try to explain to my kids, but uh Modal basically helps companies run AI. So if you think about all these cool companies out there, uh building a lot of cool AI applications, whether that's Sunno, which does AI generated music, or lovable, which powers a lot of live coding, live coding, uh they all need infrastructure.

[00:00:51]So, so we help them with the underlying infrastructure layer. And part of why that's needed is a lot of traditional infrastructure if you go to the clouds, the hyperscalers, etc. Uh they're just not built for AI, it's quite hard to work with these large models, iterate quickly, work with GPU capacity, whether that's, you know, training or inference or sandboxes or a bunch of other stuff.

[00:01:10]So we built this underlying layer that helps engineers move much faster uh and helps you kind of abstract away in a way you don't have to think about that underlying capacity routing load balancing all that stuff. So when you say that the cloud providers and I'm thinking about the the hyperscalers and then I'm also thinking about the the the neocloud companies who we call them neoclouds they call themselves AI specialized clouds but I mean their they their pitch their whole pitch is that we make it easy for you to run your applications on our cloud and so so I hear you saying that they're not actually well suited then to run the application. So help me understand why you think it is that that that your layer needs to exist and that an application can't just go straight to a cloud.

[00:01:59]>> Yeah, there's a couple of reasons like the Neoclouds are great by the way. We use a lot of NeoClouds. We work with a lot of hyperscalers. Uh we consider them partners a lot of them. Uh what what they give you is raw GPUs or or raw hardware. So you have to still set it up and typically the way you get access is you rent you know for for a year or three years or something like that like kind of fixed number of capacities. So you still have to work with like sort of fixed capacity. What the a big problem is when you have a lot of these applications in production your your demand is actually very variable. You have this like you know peaks and troughs and you know that depends on your users. So a big part of the challenge that a lot of people face when they deploy things into production running on GPUs is this variable demand.

[00:02:39]And so we help them basically get very flexible capacity. We have a fully usage based model. You only pay for the for the time the GPUs are actually running.

[00:02:47]You can scale up to thousands of GPUs.

[00:02:49]You can scale down to zero. You only pay for that time. Uh and we we offer that flexibility because we pull together capacity from a lot of different NeoClouds and and hundreds of regions all over the world. The other thing is the developer experience. So when you get the these these the hardware from these clouds, you still have to deploy the apps, set up maybe Kubernetes and Docker and and and and manage all that stuff. So you need an infrastructure team. Whereas with modal, we offer a much better developer experience. And with with a few lines of code, you can just deploy something that you know audit scales completely as demand comes in scales up scales down you can hit it very quickly. You can have research team that can deploy things to production themselves. Uh you can build very complex applications >> right so so you compete with with the fireworks the base 10's the together AI and now modal labs. I mean this is what we call the the infrastructure provider.

[00:03:41]That's sort of the the segment of the market that you're competing in right now >> roughly. Yeah, I I would say all of them and and modal what we have in common is we think of ourselves probably or at least like we think of ourselves I don't know if they think of themselves but as a kind of a second cloud layer. So so you have the underlying cloud with with the capacity and then we sort of manage a layer above making that easy to access making possible for engineers to quickly um we we we offer LM and friends as a service but we also do a lot of other stuff. So I think we're a little bit more general purpose than many other players in the space. So long term, I guess my question is why does why does the layer that you live in right now, this second cloud layer, why does it need to be um an independent layer? In other words, long-term, I mean, this is a hypothesis I have. I mean, I could kind of see the cloud companies looking to you or to together or to fireworks.

[00:04:33]Uh, you know, even the neocloud companies, they're all building out their own software platforms as well. Uh I mean I sort of see the long-term game here being a company like you getting acquired by by a cloud provider. Do you not see that to be the case in the long run?

[00:04:49]>> I I don't think that's true. I I think I look at you know historically you look at companies like Snowflake versus Redshift. You know Redshift was AWS's product. Snowflake was a standalone database. Uh Snowflake is doing exceptionally well even though they're running on a lot of hyperscalers. And we we think it's something similar here. We we're building a service higher up. The the the Neoclouds and the hybrid scoses, they're very good and we work with a lot of them. They're very good at offering capacity all over the world at a decent price, but it's quite hard to manage that capacity. But what they're very good at is putting that behind an API and and making it possible to, you know, they're very good at running data centers, physical capacity. Uh what they're often not that good at is is in my opinion delivering good developer experience and delivering a software product that makes it possible to iterate quickly. And and I frankly think of it as like a great partnership like we're building that layer above and we're you know spending a lot of money on the underlying uh hyperscalers and neoclouds. And so in that sense I think you know we we don't necessarily think of as a competitive dynamic. We think of it as a you know partnership.

[00:05:50]>> What are you planning to do with the money that you raised?

[00:05:53]>> Well, we're planning to grow very quickly. We we are hiring a lot of people. We're quite acquisitive. So we might go out and buy some companies in order to move faster. uh as well as >> what do you what do you think you what what are you looking to buy?

[00:06:06]>> We like small teams that are extremely technical and maybe haven't had the commercial success they're expecting. Uh there's a lot of companies out there that are exceptionally strong engineers building very cool things. And >> I mean I'm thinking you know I don't know like like AI coding for example that's a place where developers have been flocking to you know maybe that's an area that you're trying to build more compatibility with. I mean, you know, maybe there's more uh on the back end integrating with the cloud company like like where where do you think you could see yourself actually making an acquisition? I >> I think there's a lot of interesting stuff in in complex AI stuff, you know, whether that's reinforcement learning or LM inference optimization, things like that. Another bucket of things we're looking at would be things like file systems and runtimes and like very sort of low-level, you know, Linux and and deep sort of, you know, management of of infrastructure. a couple different buckets, but we're, you know, I think there's a lot of strong teams out there that we're very excited potentially working with.

[00:07:04]>> And so this is the other part that I have been interested about with your segment of companies is you are basically you you are renting uh cloud capacity out from the cloud companies.

[00:07:17]Do you own any chips yourself at all?

[00:07:19]>> No.

[00:07:20]>> Okay. Is that a plan? Do do you plan to get into that business at all?

[00:07:24]>> I I wouldn't rule out anything. I think everything is is on the table like we we are a software company and and you know being pure software enables us to grow very quickly and move very fast and being very capital efficient um more like a marketplace business frankly I I wouldn't rule that anything out like it's possible we might own chips in the future but it's not something we're like particularly desiring to do >> right now the developers that are your your core customer base here I wonder if you think about just the last six months and and the trends uh with respect to how they are using your platform, things that have surprised you maybe. You know, we've seen AI coding um be very top of mind. We've seen token maxing come in and out of the the um periphery in terms of what people are focused on. What has surprised you about what developers have changed in their expectations the last six months?

[00:08:18]>> Yeah, I I mean I think a big factor for us has been the massive explosion in in sandboxes. So, so modal started out 5 years ago thinking about this space as a very general purpose like we wanted to build a platform with a lot of different tools and so we we've we saw a very strong product market fit initially within friends we've been working on training as well u seen quite a lot of batch jobs uh we put out a product called sandboxes about 3 years ago and the idea is you can you can basically run third party code in an isolated safe way typically LM generated code because we saw that a lot of people were using LM to generate code uh that exploded this So we we've seen tremendous growth and enormous traction with reinforcement learning, with vibe coding, with background agents. So I think it sort of reflects a lot of the new world we're in where engineers are are running uh background agents and and building a lot more code and and in many cases under the hood, it's actually powered by modal when they're doing that or when when some companies are training coding models, they're using modal for for for reinforcement learning for the rollouts.

[00:09:18]How is the the the issues that GitHub has been having? And I'm talking about the outages here. Has that affected uh any anything to do with your business or any integrations at least?

[00:09:28]>> I mean, we were affected by it. We actually spent a lot of time like dealing with there was a merge issues.

[00:09:33]Um we one of our engineers tweeted about it and it went viral. Um >> what did he what did he say? What did he say >> that that you know GitHub is having some severe issues? I I I'm GitHub is a product that hasn't been well maintained in my opinion for for quite some time.

[00:09:50]So like I don't know like I I think that the the space would benefit from some more alternatives and some more development.

[00:09:56]>> What what haven't they done well that you think they could have done better? I >> I think GitHub is it has all the engineers. It has an amazing set of you know community and and the tools are quite good but there's like constant small issues here and there. Like I don't know. I I love the product a lot.

[00:10:12]like overall it's it's gotten us to to a very good place where I mean we I've used it for probably 10 15 years at this point u but small bugs here and there that is sort of accumulating >> and I mean for someone who I don't use GitHub so so you know I'm asking for a bit of of an explainer here like what is like h the product just hasn't innovated is that the idea or like what exactly has been um slow about it and how could they have innovated I guess >> I I think there's many things that GitHub could have done in the past I mean they very early to AI coding. I mean they they had the the co-pilot very early. I think they you know could have actually been a more dominant player in that space. I I think the product is just not caught up with all the development. Um I don't know. I I mean I again like I I do like GitHub. I don't want to you know >> it's interesting you know because we look we the other story I was going to ask you about is we published a scoop a couple weeks ago about OpenAI possibly developing their own alternative to GitHub. And this is something that that we've been kind of interested in is is companies >> uh you know finding GitHub less effective in some ways and then maybe looking to build their own similar tools the extent to which an AI native company could develop a better version of GitHub that could very much handle AI coding volumes better. I mean do you think that is inevitable? If OpenAI came up with a product would you consider uh buying that? I I think the community or the world would benefit from more choices for sure. And one one thing I just to call out like since it's somewhat related to modal like I do think with more agents writing code we will need more sort of AI suited tools. We we will need tools that enable agents to move faster as well as engineers of course too. Um but so there are some like we're we're very bullish on for instance continuous integration like the ability to scale up and run tests at very large scale which is is is a place where GitHub has been a dominant player. I I think there's a lot of cool stuff you can build over the next few years that will enable both engineers and agents to move much faster and hopefully some of those things will be powered by modal.

[00:12:12]We're very excited about the space.

[00:12:14]>> Great. Well, Eric, I want to thank you for coming on. That is Eric Bernardson, the founder and CEO of Modal Labs here on TIVv.