2026 ZKast - AI Infrastructure at Scale: Network Performance, Security & Cisco Secure AI Factory

[00:00:06]Welcome to Zcast everyone. I'm Zas Caraval from ZK Research and I'm here with Tim Shanahan who leads global sales for Cisco cloud and AI infrastructure.

[00:00:15]This is part of the ongoing series uh the Cisco blueprint series which focuses on uh real decisions IT leaders face as AI changes their infrastructure requirements. Now uh this has been the the segment that you lead Tim has been a fastmoving and equally fast growing part of Cisco's business. uh there's not a company I talked to today uh that doesn't talk about AI but you know we've come down to the point now where the harder question for a lot of organizations is really whether the infrastructure can actually support AI at scale uh you both from a performance perspective but also securely and economically and so the market conversations have moved beyond model selection and a lot of the things we talked about before in the experimentation phase and companies are now wrestling with their production infrastructure, network performance, power efficiency, SC, you know, all those things that go along with it. So, uh I think it's a great time to talk to you Tim because AI changes the data center equation is particularly the network is has a big impact on that, right? So, um before we get into that though, just a quick bio on yourself, Tim.

[00:01:26]>> Yeah, absolutely. So, my name is Tim Shanahan. I work for Cisco systems and I lead our cloud and AI infrastructure team. So these are the teams that engage with our customers that help them modernize the data center and prepare for AI and are deploying some of the most advanced technological uh use cases in the industry today.

[00:01:47]>> Yeah. And there's a lot as I mentioned there's a lot going on within the network in the data center right so we've seen more GPU utilization that of course help you need to get the data to those GPUs without creating network bottlenecks there's more east west traffic and these distributed environments we're talking now about scale up scale out scale across and scale you know everywhere um and as AI becomes more autonomous security and governance needs to be baked into that architecture and I'm curious you know you spend a lot of time with customers that's your role uh that have real AI ambitions today, but uh they also have a lot of complexity. And so from what you're seeing in the field, where are those conversations today?

[00:02:27]>> Yeah. Well, first of all, Zeus, it's great to be with you and I think you framed it up exactly correct. When we look at the customer conversations certainly that I'm a part of, there is no shortage of ambitions as it relates to AI. Every customer that I'm meeting with and our teams are meeting with are talking about how AI is going to change their business. Uh the competitive advantages that come with it. It's going to shape the way they engage with their customers differently. But increasingly to the point you made, the conversation is shifting from what can AI do for us to can our infrastructure actually support what we want to do with AI. And as you know that's a very different conversation. um you know you can have the clearest of AI strategies but if your underlying infrastructure is fragmented or it's reactive or unable to support the demands of AI around things like performance security endto-end visibility then that becomes an inhibitor for you to actually achieve the things you're trying to do and so when we talk to customers we're actually encouraging them to look at a very few five um practical warning signs that really help you wrote a blog for AI or not >> and I want to call it you actually wrote a blog called five signs your data center is holding your AI strategy back right and so dive deep that I'm curious how do I leaders actually know when their data center is the thing that's holding them back or their their network there because you know let's you call it what it is right over the last decade or so companies spent an inordinate amount of money refreshing upgrading, modernizing things like that and AI is a particularly unique beast. I think it's much different than the cloud era. So, uh talk about that blog, but then how do I leaders actually recognize that their data center is the thing that's constrained them today?

[00:04:23]>> Absolutely. So, we call this out in the blog. More importantly, when we engage with customers, this is how we engage in many of the conversations. The first step is we look at their operating model. And I would say to any customer, be honest and make an assessment. Is your operating model too reactive? And one way to gauge that is you look at your engineering team and just gauge, are they spending more of their time chasing alerts or doing the troubleshooting? Because it's not just a sign of efficiency, but it's a sign of capacity challenge because you have your most valuable resources focused in on solving complexity problems as opposed to building the platforms that are going to help you reach the goals you have for AI. Um the second one is look at your utilization. So for those customers who may have already embarked on AI, there is no shortage of use cases as we touched on briefly a moment ago. But you need to look at your utilization because that's going to tell you, do I have the ROI that I expected from this infrastructure or not? And if you don't, more often than not, it's not a GPUbound problem. It's more often than not going to be a network bottleneck and that can turn your ROI equation a Zas upside down really really quickly when you think about the amount of money that you've spent for this but the return you're actually getting.

[00:05:47]>> Yeah.

[00:05:47]>> Another important sign Oh, go ahead.

[00:05:50]>> Yeah. No, I was going to dive deep but actually go through the the signs first.

[00:05:53]>> Yeah. Yeah. So, another one is security.

[00:05:56]You know, traditionally speaking, security, I think, is recognized by everyone as super important, but it's more often than not been added on after the fact. Well, when you look at AI and particularly the traffic patterns, you have more data in motion. You called this out earlier, right? More east to west traffic, you have now with aentic operations, you have systems talking to systems. So, you cannot rely on perimeter defense alone any longer.

[00:06:23]Security has to be embedded in the fabric and so you can protect every workload, every user, every large language model um end to end throughout this entire factory. The fourth sign is look at your visibility. Do you have endtoend visibility when you look at your system and or is it fragmented?

[00:06:45]Meaning are you using multiple tools to get that visibility across your estate?

[00:06:51]In the world of AI, the smallest performance issue when you move it from pilot into production can become a very major problem. So that end to end insight becomes really really critical as you look forward. And then quickly to close out the fifth one is customers more often than not we see them treat AI um in these call normal infrastructure decisions. They have a network refresh happening over here. They have compute decision happening over here. they have a security initiative happening somewhere else. Um, they need to be looking at these not just as a refresh cycle but look at these as opportunities to take small incremental steps toward building an AI ready infrastructure for the future. Candidly, this is where Cisco's really been spending most of our time with customers is helping them build an AI ready data center so that they have the foundation of high performance compute. They have embedded security at a fabric level. They have observability or visibility end to end across their estate. And this is what we've built in partnership with NVIDIA that we call the Cisco secure eye factory that helps eliminate that complexity and allows customers to focus more of where they want to spend time, which is on the outcome and on the value of AI.

[00:08:10]>> Yeah, I like your the talk track you had around the the incremental steps. And I've I've used the expression before that um when embarking on these projects, these AI projects, you need to think almost chip shots, not moonshots, right? And what I mean by that is if you if if somebody thinks about the end state of what their data center needs to look like, that task alone could be so ownorous that it just paralyzes them from doing anything, right? And so um where if you take these kind of smaller control chip shots you you can do that and gradually you know you know build that strategy out. And do do most customers think that way or are they in this mindset of this is some big grandiose project that we have to you know do all at once.

[00:08:57]>> So I think we're seeing an evolution.

[00:08:59]What we're seeing is um more often than not early days customers were looking at this as this comprehensive end-to-end solution as a separate bespoke standalone initiative and because of that every decision was bespoke from their traditional enterprise application environment. What we're seeing though is this evolve and part of this evolution is they're recognizing that these small incremental changes and what I'll call their their daytoday or even their legacy environment can be exponentially compounded toward helping them achieve their goals as it relates to AI um if they do it very intentionally and thoughtfully. um understanding where they're at today, going through kind of those observations across the five different ways to make a self assessment, but also understanding what the end state looks like, uh and then making those changes along the way.

[00:09:52]>> Yeah. And I'm glad you have the self assessment. One of the things I found funny about those assessment tools, and uh when I when I was at a VAR, we used to use those as well is companies always overestimate the capabilities that they have until you give them some kind of tool. So, that's a that's a good thing to give them. Now Tim uh one of the things I hear over and over is that business leaders want to move much faster than the uh on AI than the infrastructure teams feel prepared to support right it's like you move too fast nobody wants to be the part the thing that fails now your blog makes the point that a customer can have a strong AI vision but still stall because the foundation isn't ready and so as an IT leader what are the this this the signals that they might have that would indicate that it's an infrastructure problem more so than a strategy problem.

[00:10:44]>> Yeah. Um so that is exactly the tension we're seeing uh play out dayto-day in customer conversations. Customers are in no way lacking any ambition as it relates to AI. What they're actually facing is what I call an execution gap.

[00:10:58]So there's a few clear signals that help us indicate that. One is when you look at the business and they've identified the value of the use case for AI. Um but it cannot consistently provision, they cannot consistently secure, they can't monitor the environment the way that they need to monitor the environment and the discussion becomes less about innovation and it becomes more about the constraints and ways you can identify that Zeus is how long does it take me to provision? um can I protect the data?

[00:11:33]Can the network have handle the workload at scale in production? And so when you begin to answer these questions, it gives you the insights to really understand where some of those underlying um you know potential challenges may exist. And so what we see is some of the highly skilled engineers are spending most of their time navigating that operational friction trying to answer the questions that I just um kind of set on the table. And so if they have to do that every time we bring a new workload or a new use case forward for AI, you can begin to see the compounding effect that it has in the organization. Uh, another one we see is when an AI projects are treated as those isolated builds that I kind of mentioned a moment ago. when when one team purchases accelerated compute, another team is purchasing the network uh or doing a network modernization project, another team is doing security. The point is is in an AI world, it doesn't operate in these separate lanes. There's much more interdependency.

[00:12:39]So the performance, the security of the workload and the dependencies and all these elements are very closely coupled together and that is another area that needs to be considered. um when customers pull all of this together and they and they go down this path and I'll say starting with modernization um they're not waiting for that future AI budget to be released or that big project come forward and what's happening is they're accelerating this journey much quicker than they probably are anticipating. At least that's what we've seen time and time again as we've engaged with these customers that are making the necessary changes around operations, around security, around uh network and visibility.

[00:13:25]All right. Thanks. That's yeah and um you know on that point you know AI infrastructure um isn't cheap. It's an expensive investment. Um I think there's been a lot of focus in media and you know really everywhere on on the GPU right people talk about the GPU the GPU as the cornerstone for AI and it's obviously very important but one of the things you point out in your blog is that the GPU alone doesn't deliver that outcome right you need if you're going to have fast processor you need fast network so um talk about why the network has become so central to AI economics and getting the ROI and And do customers understand this today? Are are you finding there's a a kind of movement in the tide where the awareness the role the network plays is, you know, a lot more um in focus today.

[00:14:21]>> I think we're on the front side of the awareness. Uh and let me explain what we're seeing and how this is beginning to play out. GPUs, as you know, they're they're only valuable when they're actually productive. And when I say productive, I'm talking 97 98 99% utilization rates. These can be pushed extremely hard. But if the fabric that is connecting all these systems together cannot keep pace, well then you just made a very very expensive investment that is running at call it 50% utilization. Um and that is what I think is the light bulb moment for many customers and they realize that it's not just about GPUs. There's much more to this when you think about AI. Um AI workloads fundamentally. Uh they they're changing the network traffic patterns.

[00:15:10]Um we talked about the east west traffic and that's predominantly being driven based upon GPU to GPU uh traffic or communication. It's workload to workload productivity. It's data moving across all these distributed systems which is only going to be compounded when you consider things like the power envelope that's being put on the enterprise data center. You're now going to see this right extend into service providers in the neo clouds elsewhere. And then of course you have the edge phenomenon which is inferencing is going to happen much closer to where the decisions are being made. What this means is the network is no longer simply about connectivity. It is fundamental to the AI performance equation. And this is especially important when customers move from experimentation into production because a pilot can tolerate some of those inefficiencies.

[00:16:05]But when you move into production, um it shines a very bright spotlight on some of those challenges and it is uh making customers much more aware about the dependencies that are on the network. Um there's one other practical dimension um which is most enterprises are coming from a brownfield environment and they're not going to be able to start with a green field. In fact, most are not going to have a green field environment. So what this means is they need a path to allow some of the new AI capabilities to work with some of their existing um operations with their existing environment with their existing infrastructure. And and this is a great testament to why Cisco believes so strongly that the high performance Ethernet is the future for the enterprise for AI type workloads. When you consider how well customers understand Ethernet, >> when you think about the operational skill sets that are built around Ethernet, you think about the interoperability for Ethernet. All of this when you pair it with the right silicon, with the right optics, with the right software, create helps you eliminate those operational silos and accelerate um your ambitions, your use cases forward much much faster. And again, these are some of the underpinnings of the Cisco secure AI factory with Nvidia that kind of come together alongside with security and observability that make it so powerful.

[00:17:35]>> Yeah, I want to unpack a couple things there. The Ethernet argument is one that historically uh Ethernet is lost because uh it just didn't perform like Infiniband did. Now, every survey that I've ever done historically pointed out that all things being equal, people do want Ethernet. We have the skills. We understand how to work with it. It's very simple, right? And if you think about the role the network plays, it's again, I brought this up before, it's not just scaling up within a rack, right? you're also scaling across data centers, scaling out across between multiple data centers, right? Ethernet does a much better job of that. Now, I think and talk about some of the innovation here with the new Cisco silicon 1. There's negligible performance differences now between Infiniban and Ethernet. And so now you get the best of both worlds, right? You get the simplicity ethernet, but you get best-in-class performance. you know, that seems like it's been a a big tipping point for Ethernet.

[00:18:34]>> Yeah, I would agree. I mean, I think if we go back a few years, certainly from a latency standpoint, um I think there was an advantage for Infiniban in certain types of workloads, but when what we've seen from a progression standpoint from 100 to 400 to 800 to now 1.6, six. What we're seeing with liquid cooling, what we're seeing advancements in the silicon, um to your point, we're seeing that become negligible in many if not most use cases to the point when you combine not just the performance but with the economics behind the performance um it tips the scale very quickly toward Ethernet.

[00:19:14]>> Yeah. Now the other major shift uh is that AI isn't just generating more traffic. It's introducing a lot of new behaviors, right? So we're seeing systems become more distributed and agents are acting autonomously, right?

[00:19:29]And agents are going to create agents, right? And so when you think about that, how should customers be thinking about the security and visibility in their AI ready data center?

[00:19:42]>> Yeah. So when you think about both of those topics, um they're no longer separate conversations or separate layers or something that you can add in later. They have to be part of the infrastructure architecture from the very beginning. You know, traditional applications um largely operated around known users and around known workloads.

[00:20:02]Um that's very different in an agentic world, right? Where you have communication machine to machine and machine speed. And so that expands both the opportunity with AI, but it also expands the attack surface. So having the right security in place um beyond just a perimeter uh model is going to be fundamental and and critical. Again, it's about the user, it's about the data, it's about the models. Um it's it's the entire thing end to end. And then being able to give you the real time insights with visibility across all of that infrastructure becomes kind of the added value of what Cisco Secure AI factory with NVIDIA delivers. Uh and you get that day one. Um the network matters so much because the network uniquely positions you to be able to observe and interact and enforce the policy. But when in when it's embedded when security is embedded in the fabric then customers can actually protect not just the infrastructure but the actual AI workloads and again those workloads are not static they're in motion all of the time. So some practical starting points that I would say a customer can take to assess their readiness. Um look at uh things like is their team proactive or are they reactive? Um and again we're going to visibility and security. Um are they able to look at their accelerated compute investment and is it being effectively utilized? Um can we enforce that security policy I mentioned across all of the workloads in all of the data flows inside of this new agentic AI world? and can we see the performance of everything from the user experience to the application experience to the network performance end to end. So when you start to look and answer some of those questions that assessment creates a roadmap for you as a customer um and it doesn't need to begin with the largest possible AI deployment. It can literally be a simple shift or or an investment in the way you look at operations, in the way that you're modernizing your network, in the way that you're embedding security beyond just the perimeter. And so customers want to see value out of AI. And now is the time to unlock that complexity. and and Cisco plays a really critical role in helping our customers do that because of how we bring together uniquely positioned security, connectivity, observability with accelerated compute for our customers ambitions.

[00:22:43]>> Yeah. In fact, I'm glad you you you finished up with that because there's a lot of AI factories out there, right? I mean everyone's got an AI factory and most of them focus on network and compute and it seems to me that security uh is becoming I think a little bit better understood but observability has been almost a missing piece of this and there's that whole expression you can't secure and you can't manage what you can't see and to me security performance and you know security network and observability are all I guess uh and if you had a three-sided coin right they'd all the same, you know, the same uh different sides of the same coin here. And uh I I think um you know, observability is probably uh the most underappreciated aspect of AI factories today because I think everybody understands the the infrastructure uh requirements in that.

[00:23:36]But um you know, just to finish up on that, I guess.

[00:23:40]>> Yeah. So, I'll say this. Oftentimes when we talk about observability, the reaction we get is we have monitoring tools or we have tools that give us visibility. But when you really begin to unpack that with a customer, what you realize is they have a long list of disparit legacy tools that they're managing their environment. Um, but they're not built for AI and the next generation of how you monitor this environment. And they're not um cohesively integrated with one another.

[00:24:10]And so what we're talking about when we say observability, it's really one tool that's managing this entire environment that's collecting the data, providing you a single dashboard that allows you to look at the user experience, the infrastructure utilization, you know, all the individual elements that's also tightly integrated with security and security operations as well. So again um when you see it in operations um it is a very unique experience and it's one that is helping accelerate our customers journey. It's taking the the complexity out and accelerating the time to value um which we're super excited about and very encouraged by what we're seeing with customer deployments.

[00:24:53]>> Great. And uh so on that note, I think it's a good place to wrap up. Uh uh I appreciate the time. Tim, anything else you want to add?

[00:25:01]>> No, thanks for the time. Um, we would love to engage with any customer who's considering this. Whether you're at the front end of a journey, you're midway through a journey, or you're having challenges with your AI journey. We're engaging with all three types of customers. And would be encouraged to sit down with you and share with you some of what we're learning about our own uh journey ourselves, but also what we're learning through other customers and how we can help you shape your future uh on AI and make it a successful one.

[00:25:29]>> Yeah. And I, you know, in wrapping this up, I I do think the AI conversation really has entered a new phase, you know, this year going into next year.

[00:25:38]Customers are no longer evaluating AI as an innovation opportunity, but actually evaluating whether their infrastructure can sustain it. There's no question everything we do, right, AI is going to be infused in that. And so the time to deploy is now. Now uh a couple of recurring themes from our conversation is that AI readiness um needs to be well understood and so you have to think about whether your your team has the right skills or not. Um uh whether accelerated compute is being practically uh or you know productively utilized whether security is built in or bolted on. I think that's another strong consideration. I think the don't forget about observability because you have to be able to see across that IT environment and um I think the last thing to keep in mind is whether your modernization investments are aligned to the future workload strategy. I think the worst thing a company could do now is maybe go through some incremental change that isn't pointed to this world where AI you know is going to be um uh you know pervasive and then you wind up having to rip and replace. So I do think Cisco's position is that um around their AI factory of network and compute security and observability all need to work together is probably the most salient point from this conversation. So on that note Tim I want to thank you for your time.

[00:26:58]>> Thanks Zas. Appreciate it.

[00:26:59]>> Yeah. So so on behalf of Tim Shanahan from Cisco I'm Zas Caraval from ZK Research and thanks for watching. Uh give us a like and also hit that subscribe button. We'll see you next time on my next episode of Zcast.

[00:27:12]>> [music]