Are AI Gains Worth the Cost?

Added: 2026-05-14

[00:00:00]Well, uh, it's been a while since we've had you on the show and a lot has transpired in the world of software, AI, and so I want to get your thoughts on it all. I want to start with, uh, look, these layoffs that have been sort of smattering through the last couple weeks. I mean, we've seen it started with Block with their big layoff, right?

[00:00:19]And then we we've seen other companies uh big companies like Meta uh Coinbase you know Cloudflare all these companies reductions in headcount AI is central to it. What's your reaction to it?

[00:00:32]>> Yeah I mean um I think there's a couple different uh probably components to to to them. O obviously it's easy to to sort of have them all all kind of point back to AI as a as an you know as an underlining under underlining kind of point. Um, uh, as I've sort of looked through some of the memos and the announcements, I think you have in some cases, you know, there's probably some companies that maybe had more more staffing relative to, uh, to to where their business model was. Um, in other cases, I think you've seen situations where it's a bit of a retooling of the organization. And I don't I don't you know know that I can speak for Cloudflare, but I've seen examples where Matthew has suggested that actually they're going to be hiring just as many people, but but it's a different set of roles that they need to to be able to hire for. And so you kind of have AI automating uh or augmenting work in one area and then reallocating dollars to another area that that might be, you know, more strategic or higher growth. I think as a general matter, companies are kind of always doing sort of that type of of sifting through that resource allocation and deciding, you know, is this a year we're investing more in in one particular domain or job function versus another. Um, clearly it's it's all all sort of wrapped up now in in AI as being the the kind of cause of that.

[00:01:46]Um, but I I think there are certainly areas where where companies will say, I'm going to invest less incrementally going forward in one particular pocket.

[00:01:54]um and then take those dollars and either some of it will go into compute as an example. So we're seeing some companies just you know really have to be able to go fund the token expenses or then take those dollars and reallocate them to other higher growth areas I think as we've seen in maybe the Cloudflare example. But um but I that that's how I would sort of you know you know think about and characterize it.

[00:02:13]That's not what we're seeing or doing in our business and I have lots of friends that are >> I was going to ask you but tell us about how you're managing it at at at Box.

[00:02:20]Yeah, I mean on our end there's there's definitely uh uh areas where where on uh on an incremental basis the dollars are going more into one org versus another because we might be getting more leverage from AI in in one spot taking that leverage that we're now getting and then reallocating uh to another function. As an example, um you know we are hiring uh you know meaningfully in areas like go to market. Uh so if you think about you know sales reps or customer uh uh support or you know customer success managers or consulting there's a as AI rolls through the economy our customers need a tremendous amount of of help and support of how do you actually adopt these tools? how do you enable them and embed them in your business processes? So, there's actually going to be more people needed than ever before for that side of a software company um when it's actually getting rolled out into an organization, right?

[00:03:12]>> Um and uh and then there are other areas where we've decided okay maybe like the incremental dollar again has to go more into some of those functions than than a different one. Um but I think on in an absolute basis I I can't think of many orgs that will be sort of smaller in total size in two years from now. Uh simply because there's there's just still so much to do uh within our organization and uh and the things that we have to go work on. I >> I mean you mentioned consulting and this is one of the one of the things that among the many uh things that you've written on X you know your points of view on software and the sector. uh you had a post last month actually that caught my attention about I think it was open AI partnering with consulting firms and you made the point saying look anyone who thinks that uh companies are just going to do away with people and know how to implement AI I mean take this as uh as a as an example that that's not going to be the case. Um, you know, I do want to ask you about another post you put out there. I'm going to read this part of one of your expos. As agents become the biggest users of software, then all software has to be available in a headless fashion. Agents won't be using your AUI. They'll be talking to your APIs. Can you expand a little bit on what you meant there by a headless fashion?

[00:04:28]>> Yeah, I mean, this is the this is the, you know, one of the I mean, there's like multiple trillion dollar questions right now in technology. This is one of the trillion dollar questions which is which is in a world where you have AI agents that you either deploy in some kind of completely background fashion where you or I never see them as end users. They're kind of just always on and running or where you or I go into codecs or co-work and we ask a question and and the agent goes off and fans out across systems and does some work for us. In both of those modes that I just mentioned, um, we as users of of the the agent or the the underlying tools never see a user interface for the software that that agent is consuming. So, um, so we don't we're not we're no longer pressing any buttons. We're not clicking on any uh any parts of the interface.

[00:05:12]And the agent certainly doesn't need to either because the agent is able to sort of say, "Oh, I understand the API of this company. I understand the the structure of of how the data models work and I can go and interact with this in a in a kind of a a an API interface. So in that world then the the sort of typical way that an enterprise software company thinks about its monetization uh the way that that we go and build our our very technology starts to subtly shift because um you might have an agent that could do the equivalent of what maybe a thousand people would have done within a single seat. out. So imagine if I have an agent that's sort of just constantly sort of combing through data or processing information or executing tasks um and I I let that agent run in parallel and 24/7 and it can work at kind of hypers speed. Well then obviously that wouldn't make sense as sort of a single seat of of that software anymore. you have to have a new way to monetize that that represented all that utilization of of the the the software >> which is sort of like the this whole usage based pricing uh movement.

[00:06:17]>> Yeah. So so I think the the way that this plays out and kind of what I've laid out um is and it's not particularly like you know um that that crazy or or counterintuitive. It's it's kind of the natural evolution that that one might expect which is for you or I uh we'll probably continue to have seats inside of a lot of the software that we use.

[00:06:34]like we will still have seats inside of Slack and inside of Box and inside of Salesforce because there's a lot of reasons we have to go into those tools and do work ourselves as well when agents are kind of working on our behalf and they're doing uh let's just say a kind of a normal you know amount of work for us maybe it's like three times what we currently do but it's not a thousand um I think I would expect that most software vendors have to let those agents use our seats um in a completely kind of included fashion so we can go and have an agent go and do work for us again in codeex or co-work and it will execute some tasks and I I think most software companies will sort of say okay that's actually something included in your seat price um going forward but then you're going to have a lot of agents that either do an insane amount of volume of tasks well beyond what what you or I would be able to deploy you know just a few agents for or you'll have scenarios where it doesn't make sense to have a seat because the task is not sort of owned by one individual it's sort of owned by the corporation it's a it's a it's a background process that is sort of always on or happening um that uh that we're executing. And so so if you think about that kind of work, that should almost all be consumption. So um you should just basically say, hey, if I have an agent do 5 million things inside of Salesforce, I should pay for the equivalent of 5 million API calls or agentic API calls or whatever that looks like. And so I think you're going to see a new component of the business model of enterprise software be what is sort of the headless API usage of these systems.

[00:08:00]Now, some software companies are already sort of built for this, by the way, because we've been doing paz um uh uh business models for a while, which is platform as a service where you can already use our APIs in the background inside of other software. Agents just are sort of a force multiplier because the the scale of this could be far larger than any amount of kind of pad usage we've ever seen. Well, so let me ask you a question then. So, in this usagebased model, which look, this is nothing new. I mean, usage based pricing has been around for a while. I mean that's one way that enterprise software companies uh will be able to keep people paying for their products based on how much they use. The the flip side to this question though is I mean you know in terms of pricing right now uh pricing right now in a lot of ways uh certainly for newer companies it's subsidized.

[00:08:46]It's lower than the actual price will be and we you know we've reported here at the information of about uh companies raising their prices not the least of which is anthropic. you mentioned co-work uh you know is something that you guys use. So I'll talk about your your own experience in a minute which I'm curious about but just broadly speaking I mean when prices do go up you know what evidence do do we have that people will still keep paying for them?

[00:09:14]>> Um well uh I would say this is sort of unrelated point to the headless API side. Would you just so I'm clarifying right?

[00:09:22]>> Yeah. Yeah. I'm talking I'm talking now we're talking I mean consumption based pricing is one thing I'm I'm now talking about just pricing as a whole as as a separate >> topic. Yeah. Well well because I I think and this is going to be these are going to get lumped together a while for a while. Consumption pricing can be can be uh there's two categories of consumption pricing. There's consumption pricing on just the API calls of our underlying services. You know the API call into box the API call into workday the API call into Salesforce. And um and that's a sort of a predictable pricing model.

[00:09:50]it's it's mostly hitting CPUs um uh or or you know kind of storage environments. So so we we kind of know the how to model the cost structure of those types of API calls. There's a different part which is the subsidization of tokens which is you know much more of a constrained resource right now that's obviously a GPU capacity and and the kind of surrounding components um and uh and there has been you know some business models that have been subsidized uh where uh where the underlying you know kind of um you know tokens are are being paid for by by by the vendor obviously to try and get market share or kind of you know prove out use cases. um then now ultimately probably you know prices have to to rise um uh or they're rising as a result of scarcity and so then that's just market forces playing out which is you know if you have a scarce resource you can charge more for it and then you kind of find like what is the market clearing price for where you start to you know lose customers at a rate you know higher than the the the sort of revenue you you get by raising those prices. So I actually think we're we're just sort of seeing microeconomics play out you know at a at a big scale. Um I'm not I'm not I don't think there's any kind of meaningful you know thing that's happening other than that which is prices will rise up to the point where volume um you know sort of drops and um at least the rumors I've seen the gross margin on inference of the major Frontier labs is actually like very very high quality gross margin from an from just an absolute number standpoint. So so obviously you have to pay for things like the the training costs, you have to pay for things like the infrastructure buildout. Um but um but I I don't think this is uh I mean you can see in Anthropic's topline numbers and OpenAI's topline numbers clearly clearly we keep paying uh for uh for the AI which um is effectively revealed preference that it's working and and we are adopting these tools and you know if you go to most engineering organizations you would never be able to rip out AI from them at this point um simply because the productivity gains are are too high. I I'm I'm I'm realize that asking you this next question is sort of asking you to pick favorites in some way, but co-work uh you know cloud code uh codec which which suite of of products are you using uh more right now?

[00:12:02]>> Uh you know the battle of our lifetimes.

[00:12:04]I have uh I have two icons right now on my Mac tray uh uh for uh for each of those. So uh and I I like to be pretty ambidextrous um uh partly for my own use cases but also because it's really important that we at Box understand the the ways that the people will be working in the future and we want to make sure that our platform works extremely well with both of those as well as others. I mean I have perplexity computer running in a tab uh often times. So there's there's you know three five 10 tools that I'm I'm personally using. I I unfortunately will not be able to uh to pick a favorite amongst those at the moment. But um I think we're we're we're in an incredible moment because of the pace of innovation and the the sort of pace of of product development that's happening um where ultimately us as customers and consumers and proumers are just winning because of how much innovation is happening.

[00:12:50]>> Well, okay. So, and so maybe let's go back to sort of your own usage and the levels of usage then of these products and all sorts of products. I mean there is a question around uh how much budget you can allocate as a company to using these tools >> and we've heard uh you know I think it was the CTO of Uber told us that we've kind of hit our budget now for for using tools in some cases >> um and then you contrast that with other companies that token maxing is is the is the alternative. So I am curious around um did you go about setting a a defined budget uh at box and to have you blown through that budget yet?

[00:13:32]>> Uh you know I we're only a quarter into our uh our our financial year. So I probably can't talk about you know where we are in our budget um >> just just for AI tools just just for for I I I very get the I very much get the question. Um, so here's what I'd say.

[00:13:48]First of all, probably only a few companies on the planet can can afford to sort of token max. Um, so I think I think that might be like a a a great fringe benefit at at Meta and maybe a couple others. Uh, mere mortals uh, you know, have to uh obviously have to balance um, you know, the overall annual budgeting process and and how we allocate spend across the organization and um, and that's the uh, that's the kind of stuff that that you know the rest of us have to do. Uh we are um uh I think I think maybe the the the the bigger picture thing that will be super interesting is right now um uh you know AI probably started first as an IT expenditure and so that meant that AI sort of had to fit in within uh 3 to 7% of your corporate sort of of of spend as a percentage of revenue. you know, it kind of runs at three to seven percent of revenue and and you know, across the economy, some some businesses less, some more. Um, that that's probably too constrained of a of a line item uh to be able to really get fully augmented, you know, productivity across your organization. So, what's going to happen and I think and the interesting thing to watch over the next couple of years is is the jump from when AI moves from an IT expenditure to an opex across the the general sort of budget planning cycle in an organization. and when you when you as a department head have to decide, you know, do I want to add one or three or 5% of my budget to AI compute or maybe even more um uh and and what that looks like from a resource allocation standpoint. So this year we got ahead of that a little bit in especially engineering but certainly as you have things like codeex or co-worker others that are pretty you know AI you know compute intensive you'll have to do that in other areas of knowledge work you'll have to decide you know um what am I going to do in my marketing budget when when I want agents running and producing a lot more marketing collateral um what am I going to do in um uh in legal or financial financial operations when again we have we have way more of this augmented uh workforce so I think I think this is a trend that's going to start to shift. We're probably even going to need new software just for that problem because because it's it's you know in normal resource allocation. You have these sort of one-time fixed expenses which is like onboarding the new person and then their new salary that that is sort of sustained. Um in AI compute you kind of go up and you go down. You can kind of quickly tell somebody to spend less. You can tell a team to spend more. So there's a much more dynamic sort of form of how you do budget processing um you know with uh with with AI. And and I I don't think a lot of our kind of existing tools probably help with that yet. Um and and then who owns this? Does the CFO own it?

[00:16:21]Does each organization own it? Does the IT leader own it? Um many fun questions that are that are going to be uh that we're going to be confronting with over the coming years on this front.

[00:16:30]>> Right. I I just want to go back Erin before I let you go back to that pricing discussion because I just wanted to make sure I understood what you were saying correctly. I mean, look, my sense was as we've heard about companies starting to raise prices for AI related products. I mean my sense was that was a reflection of them uh you know really trying to uh make the price more reflective of what it is costing them to produce the service and I mean it's not just anthropic you know we've seen other companies also uh suggest that they will raise prices and so I mean if I heard you correctly I mean were you suggesting that basically the margin is healthy enough and they won't have to increase it as much or the question I was trying to get at is is What what evidence do we have that people will still keep using these products if the prices keep going up?

[00:17:18]>> Well, I I I would just say as of uh you know, May 8th uh or or whatever. Uh >> we're recording this on Friday, right?

[00:17:25]Yeah.

[00:17:26]>> Yeah. So, um uh as as of early May, uh it is uh it's very clear that we keep paying uh for these these tools. So, so and and I I don't get the sense that prices are going to double or triple from here.

[00:17:39]um you know you it's important to also have the backdrop of how compute constrained uh as a factor that this all is uh is driven by. So if you imagine a world where there's vastly more compute, if you imagine that you get more efficiency in the actual sort of AI uh at the software layer of AI and uh and you get hardware efficiencies in in next generation, you know, kind of Nvidia or TPU or trrenium um you know kind of uh uh compute approaches, I think start to >> you have multiple factors that can bring down the price of AI. Not to mention there's this really great kind of counter pressure that will always be out there at least for the foreseeable future which is open source. And so open source kind of is this nice kind of you know lever on the whole on the whole space which says which says at any moment I can peel off and get you know sort of frontier as of 3 to 6 months ago and be able to run that on the lowest cost stack that I can go find. And so as long as open source is around and and kind of out there you do have this nice this nice you know again thing you know uh rip cord that the customer can pull.

[00:18:46]Now not not all customers are sort of savvy.

[00:18:48]>> Are you are you using open source at box?

[00:18:50]>> We are but um but not for the frontier type capabilities. So we use open source and things like our embeddings models or like things that are really kind of background uh components of our AI stack. uh for the frontier which is I want to be able to go generate a PowerPoint presentation or I want to be able to you know process a loan document and extract the critical data. we do need models like GBD 5.5 and Gemini 3+ or um you know Opus uh 47 like these are the model families that that are that we work with um at for those types of use cases but there's lots of stuff that over time you will be able to peel off and and you know put into a uh an open source or a much cheaper model and um and we even stratify within our use cases where something will go to Gemini flash as an example um because we have you know obviously much far much lower cost tokens and the use case can be sort adequately adequately or or very very successfully served by those models. So you know in five years from now you will have when you look at the the sort of token mix of a company it will not just be that all the dollars are going into the frontier at that particular moment.

[00:19:55]You'll see a full you know kind of pigraph of of where the different use cases sort of land just as actually knowledge.

[00:20:02]>> What what does your pigraph look like if you were to say open source versus frontier? I mean, you know, I just give us a view of of what Box's operations look like right now.

[00:20:12]>> Uh, today it's probably 80% Frontier.

[00:20:15]Um, and but but I think what would happen is if I if I had to guess in in 3 to 5 years from now, whether it's open source or from one of the frontier labs as a lowerc cost model is sort of immaterial. Um, it it's it's all that matters is just like what is the cost of the token? I think you would imagine that in five years from now it would be much more like you know 30% of our of our uh token spend goes to the the truly frontier type model and then and then sort of from there you're like okay you know another 20 to 30% goes into into you know some some just behind frontier type class and then another 30 40% goes into these you know the these models that are just doing you know easier compute operations for us in the background. Now again, what what's what's amazing about AI is that lower tier model that I just laid out as an example is probably three times better than the best frontier models that we have today. So so like >> because everything will be Yeah. But you're not you're not saying 20% open source. You're saying 80% frontier and in that 20% is something that is maybe a little subfrontier and then open source percentage is probably even smaller than that.

[00:21:22]>> Well, today I was I was sort of giving you the rough allocation in the future.

[00:21:25]Uh I I just think it's you know probably I would just for for lack of any better way to think about I'd probably cut it in thirds and and probably the the second and thirds would be could be solved by open source or or again if you're a lab and you want to stay very competitive you probably would have an equivalent model that you're running at just a lower margin and that's that's your way of sort of making sure you don't get flanked by by some other provider. So I think that's sort of how I'd expect to play out. Now, a great example again of of a company already roughly delivering this shape of of strategy is Google where Gemini Flash is meaningfully cheaper than Gemini Pro and and you use it for different totally different use cases. You're going to do your orchestration and your planning with something like a Pro model. But if I wanted to do really really high volume data classification where I have to like look at you know 10 million documents that are being piped through a system very quickly a flash model or an instant model or a thinking low model you know each each vendor each frontier lab has a different way of sort of framing this that'll be totally more than adequate for for that type of process. And so by volume of token spend you'll just allocate the the tokens differently based on the type of workload. And again, it's it's sort of not to anthropomorphize this too much. It's kind of how a company already is structured. You have some things that you say, okay, this is the very rare, highly complicated task, and we need to put as much of our kind of, you know, capital from a human capital standpoint into that in that area. And then there's other areas where like, okay, there's a little bit more high volume. It's not as competitive of it's it's price competitive, you know, for the for the buyer of that of that type of task. And and then the costs are a little bit lower, >> right? Great. Well, Aaron, I want to thank you for coming on as always. That is Aaron Levy, the CEO of Box here on TIVv.