It Depends 108: Data Summit 2026 Boston Keynote - Why the Data and AI Stack Is Collapsing

Added: 2026-05-16

[00:00:00]This morning, I am very, very happy to kick this off with our morning keynote for Thursday with my good friend Sanjeev Mohan, long time friend, industry analyst, luminary, book author, publisher with a new book coming out.

[00:00:18]We were hoping I was hoping it was going to be ready this month, but it's going to be out shortly on some of the topics here, data foundations. That's a specialty. I've always looked up and respected Sanjeev. So, I'm really looking forward to his presentation this morning. So, Thank you. I will hand it over to Sanjeev.

[00:00:37]Hi.

[00:00:38]Thank you so much.

[00:00:39]>> [applause] >> I see so many familiar faces here. Thank you for coming. It's such an honor to come and present to you.

[00:00:51]The topic that I want to talk about, the title of my keynote, is the stack is collapsing. The data and AI stack that we have worked so hard over the past decade is literally going to be transformed. It is already happening.

[00:01:07]And so, today I I want to present to you how this stack is collapsing on itself.

[00:01:14]It is and what is taking place. All of it, as you can imagine, is because we are now in this era of generative AI with agentic. By the way, how many of you believe in agents? I'm just curious.

[00:01:30]Oh my god. Like literally less than five hands in a room for it.

[00:01:34]Which is which is actually great because that is exactly what I want to address.

[00:01:38]I don't want to be this blind AI optimist, which actually I am, but that's besides the fact. And so, I want to like talk to you about why what's my enthusiasm and what's changing in this space. So, with that, I want to get this the most boring slide out of the way.

[00:01:59]I uh I left Gartner 5 years ago to start an independent data and AI advisory. Lo and behold, chat GPT happened in in 2022 and now I feel like they say if you do something that you really love, you'll never work another day in your life. That's exactly who I am. I work 24/7 now.

[00:02:22]So, because this technology is changing all the time, but it's a very exciting moment to be in.

[00:02:30]We will see. I'll walk you through this.

[00:02:33]And I John, thank you for mentioning my book. In fact, a lot of what I'm talking about today is actually based on the agenda, the book chapters.

[00:02:45]My book is coming out in about 2 months.

[00:02:48]Wiley's publishing that book. It's called Designing the AI-driven Data Foundations.

[00:02:54]So, since only five hands went up, let's talk about history. I'm a huge believer in history. How many of you love history, by the way?

[00:03:04]Show of hands.

[00:03:05]Okay, so far more than believers in AI. I I think history is fascinating. If you If you understand history, it puts things in perspective and you start seeing things with a different hat. I'm going to ask you a very interesting trivia question.

[00:03:26]Mr. Watson, come here. I want to see you.

[00:03:30]Who said this?

[00:03:33]Not Sherlock Holmes.

[00:03:35]>> [laughter] >> Yes, one.

[00:03:42]Alexander Graham Bell, almost 150 years ago in March, at this site where we are, uttered those words to his assistant Thomas Watson.

[00:03:55]And that recording took place here. And if you don't believe me, when you step out of the hotel, right across in front of the hotel, there's a plaque that that says that. So So I feel like we are in that historic moment. When electricity first came out in 18 1890s, so at that time people had no idea what to do with electricity. So what they did was they had these factories cotton threshing or whatever it was, they just took that coal-fired motor and they just plugged it into electricity source. And guess what? They're like, "This is useless. What's the point of this? I'm not getting any productivity."

[00:04:37]Literally, it took 30 years. They had to tear down the building and they had to rebuild a single-story instead of a high-rise like factory kind of a thing.

[00:04:47]They had to do that. When automobiles first came out, you know what people were using it to plow fields?

[00:04:54]There were no roads. Why would there be roads? Because we were in the era of horse-carts, buggies. So no one had built the roads for the car. So the point I'm trying to make is that it takes many decades for things to to become productive.

[00:05:13]In fact, when IT first came out, here's another interesting historical fact.

[00:05:19]When in 1940s, 50s, when computers were starting to come out, there was a discussion amongst experts, what should be the future of computing? How big will computer era be? And what should be the underlying architecture? Should it be a calculator or should it be a neural network? Well, calculators were coming out, neural networks were still fuzzy like they are today.

[00:05:47]And so they built computers to be these giant calculators. That's what today's computers are. But in the future, 20 years from now, so if John invites me back in 20 years, then we will see what happened to the AI era. So it's too soon to either dismiss it or to be overhyped about it. Even [clears throat] in 1980, there was no productivity. The productivity of computers came only much later when they started changing the org chart to to be very close to to how computers were supposed to work. How many of you remember that we used to joke about this only like 15, 20 years ago that the boss gets has an email.

[00:06:37]The admin prints all the emails and gives it to the boss to read.

[00:06:41]Right? And now I mean that So So my point is that that we are in the midst of our third innings and we are thinking that oh, I don't know how this is going to end. This doesn't look too good, you know. But But the fact is that we are too early and we don't know how this will transform. In fact, almost 61 years ago in early 1960s, Time magazine raised an alarm bell because the number of computers in in the world had gone from 100 to 200,000.

[00:07:13]And so this is a cover of Time magazine edition where it basically predicted that human beings are going to be replaced by computers and so get ready to do leisurely task like backyard landscaping.

[00:07:29]I don't know how many of you have given that to computers.

[00:07:33]That's not how it panned out.

[00:07:37]In 1980, AT&T hired McKinsey and said, you know, this this this idea of cell phone, mobile phones coming out, help us determine should we be in this business?" And McKinsey said, "Ah, don't worry about it. It's just a fad.

[00:07:55]It'll go away." And so, AT&T didn't invest.

[00:07:58]And and McKinsey was wrong by 100 X because by 2000, the number of telephones had exceeded 100 million.

[00:08:08]Now they're 100 billion. So, so they they ended up buying a company because then they had to scramble to And then then there are some folks I see with Watson X t-shirt from IBM. So, IBM's chairman Thomas Watson, I think, same name. Yes.

[00:08:27]So, we have three Watsons happening here. Sherlock Holmes, Graham Bell's assistant, and Thomas Watson from IBM.

[00:08:33]All by coincidence.

[00:08:34]So, Thomas Watson was supposed to have said, although IBM says this is all made up, "The world does not need more than five computers."

[00:08:45]And that was the vision. So, so my point is that that history teaches us a lesson, and the lesson is that we tend to overestimate the benefits of a new technology in short term, and we underestimate the benefits in the long term.

[00:09:07]And so, we are in the short term where we have a lot of expectations, and sometimes meet some expectations, sometimes does not. So, so we are in this weird state of being too early in this phase.

[00:09:20]So, what I want to talk to you about is I've broken down my my talk into six sections. Let's start with some foundation, which is the basic What are the basic building blocks or basic strategic movements that we are witnessing in our industry and then we'll go through the the stack.

[00:09:41]This is a stack that I talk about.

[00:09:45]Today we're not going to talk about the the last layer. One of the things I I also do is I blog quite frequently on medium on my website sanjmo.com. You can there's a blog section.

[00:09:59]Two weeks ago I was at Gartner sorry I was at Google's Cloud Next event so I wrote quite a bit about computer and silicon. Today we're going to stick to data infrastructure and agents.

[00:10:11]So with that let's start with a foundation landscape. So what are the the big things that are changing?

[00:10:20]So we have been very myself included. I am enamored by technology. I love technology. It's but being technology first is not how the world is moving. I remember those days when I would go to an event and I'd come back and I would my management would ask me okay so we paid for you to go to this event what did you learn? I'd be like oh my god there's a new graph database.

[00:10:44]And people would be like what? It's like who cares?

[00:10:48]And that's the era we are in. Business is is driving AI not IT. IT is actually responding they're reacting to it because they're they're on the hook to deliver. Business is wanted so it's all about outcomes these days. We will talk more about this because even the pricing of products is now changing. How we price not per seat or or per compute unit. So but we'll talk more about it. The second thing that is happening is if so so the the transition that's happening is that everything we built up to this point has been for human users.

[00:11:30]And what's happening where we are it's not humans, it's the agents that are consuming and we we are the ones who are benefiting from it, but it's it's like when when Google came out with search engine.

[00:11:47]Before that, AltaVista, Lycos, you remember Excite uh that era, right?

[00:11:51]Yahoo kicked it off. So, we were so excited when Google came out. We were like, "Wow, I can so fast I ask a question and it shows me all the links."

[00:12:00]But, did I ask for links? No, but we took it for granted till AI came and then AI said, "Well, who cares about the links?" You have a question, you want the answer.

[00:12:10]So, then all of a sudden Google went from being the coolest technology to being like method to the means. It's like, who cares about about links?

[00:12:21]And then chatbots and all became the the the the de facto standard. Now, we even moving away from that. Now, we are like, wait, I asked you a question because I have to perform a task. Why are you giving me an answer? Why don't you just perform it on my behalf? So, you see how quickly we are our mindset just resets? So, so now if agents are going to do the work, then agents don't need data only, they need context. They need to understand what is my intent and then base I I and we'll by the way walk through what exactly an agent is because there's so much confusion.

[00:13:02]Every event I go to, every slide starts with agents, ends with agents and then they misdefine agents. So, so we will demystify agents. But, but you can see semantics, ontology, knowledge graphs, all of these topics that we have wrapped grappled with for decades now are suddenly very exciting topics.

[00:13:27]So, the idea is that as the stack is being reformed, we are no longer wanting to make copies of data and say that George is the data engineer, here you go, and then Julie is a data scientist, and she gets her stack, and and then somebody gets the business analyst have their own stack. We see there's a massive convergence happening of of use cases, roles, all based on the same stack. And then finally, if everything is collapsing and into one, the the main beneficiaries today, actually there's a crisis in Silicon Valley with small software companies. They're They're finding it harder and harder to survive because the integrated stacks from hyperscalers and mega vendors are starting to build everything.

[00:14:23]Just this week, on Tuesday, I woke up in the morning, I see my phone, SAP is buying Dremio. I'm like, wait, SAP was a very specific HR ERP, that kind of stuff. Why are you buying Dremio? It's like, so then everybody all these mega vendors are buying all these companies, so they all have started to look exactly the same.

[00:14:47]So then the question begs, well, who who wins in this? And that is the user experience. So, user like, how easy is it for me to get my work done? So, user experience becomes really the defining factor in this. What's easy gets adopted. In fact, um I've seen this all my life.

[00:15:10]HTML came out and World Wide Web was instantly adopted. But before HTML, which is a markup language, there were markup languages on Unix. There was SGML and LaTeX and all of these things. they were not going anywhere because they were too complicated. So, so simplicity, conversational agent, so that's what's user experience makes a big difference.

[00:15:35]Okay, so now let's get to the the actual stack and talk about data stores. So, I've divided this into operational databases and analytical databases because there's several things happening in this these two areas. So, operational data stores um uh I I the long forgotten workhorse, bread and butter of every organization. I remember in my Gartner days, we used to do this this end-to-end life cycle diagram. We used to have data coming from operation systems, and then we would forget, we would go through ETL and data operational data stores, and data governance, data catalog, BI, all of that.

[00:16:19]Tiny thing on the left is where the company's entire bread and butter comes from.

[00:16:24]Uh which could be your retail, point of sale, it could be uh your help desk, it could be any operation system. That layer is not becoming extremely important. And the reason for that is because agents need or AI needs real-time data to make decisions. And like I said, we'll go more into agents and appreciate that uh when we get to generative AI section. So, so uh it's it's now not considered to be the best option to take data from different places, uh spend $2 million. In fact, my original data warehouse projects took 2 years to complete. 2 years later, when we launched it, how many of you were in that phase? When a data warehouse took my it was multi-year project, and by the time you launched it, the business said, "Our requirements changed while you were working on it. So, thank you very much.

[00:17:20]I'm going back to using Microsoft Excel."

[00:17:23]So, so now we are trying to to leverage data as it is produced.

[00:17:30]Uh in fact, when um I I started my career at Oracle and then uh in early 2000, no SQL databases came out and there was this very cool concept called polyglot persistence. The idea of polyglot persistence was that you have the best database for your use case. If you're doing networking, kind of like, you know, relationship, then have a graph database. But, if you are doing a product catalog and products can have any kind of attribute, use a document database. If you want to log analytics, use a wide column database like Cassandra or something.

[00:18:08]But, for financial transactions, use relational database. That concept is no longer uh where the focus uh needs to be because who cares? Businesses don't care where this data is coming from. All they they want is that whoever's acting on their behalf, let's say it's agent or even if it's them, they want that whole thing to be abstracted. So, that's the first point that that uh And now you see the rise of multi um multi-model databases. So, one database, it has different uh data models in it. Even if it does not have different data models, there is some sort of a sinking happening behind the scenes like this zero copy uh stuff. So, so you get everything in one place.

[00:18:55]And then um and then uh the other thing is that I see at the bottom, unstructured data is now the first-class citizen. So, everything that we've done up to this point, literally almost everything we we've been building has been for structured data.

[00:19:12]But, where is 90% of of in organizations data? It's unstructured. By unstructured, I don't mean just documents, emails.

[00:19:21]It's like if you are, let's say, a product company, or let's say you're in customer service, and you get a call, you want to look up who this customer is. You find it in your CRM database. You want to know what did they buy? You find it in your database. But, then they say, "Well, I have I have shipping problems." Those shipping documents are in a PDF.

[00:19:44]The customer may have tweeted something about the product and the company. So, there's sentiment analysis happening.

[00:19:51]That's in in tweets. So, and then there could have been email exchanges. So, the data is spread very It's It's widespread. But, we built everything only for our relational structured or non-relational structured databases.

[00:20:07]So, now what we are saying is, "What if we took this data in near real time or real time, embedded it, and then we did semantic search on the top of that? How How powerful that that process is?" So, that's other reason why operation databases are becoming big.

[00:20:25]Another final thing that I want to say is is that these databases are becoming ephemeral.

[00:20:33]This concept literally I Even now I just find it very disorienting.

[00:20:40]To me, a database is a persistent store.

[00:20:42]It's It's super important part, and you you don't buy databases all the time, and you store data persistently forever.

[00:20:50]And here, Neon gets bought by Databricks. Why? Because 80% of databases are now now created by agents, and they come and go, and they go. And And the reason why this is the kind of operation databases we are going in is because when an agent is working on behalf of a human being, that agent is going through multiple chain of thoughts.

[00:21:15]Is should I do this? Should I do that?

[00:21:18]Every one of those thoughts is generating more data.

[00:21:22]I Yesterday someone was telling me that research shows that the amount of data that's going to go into operation data stores in the next few years is going to be 100 times all the data that has been ever created. Because agents are now creating their own data, and then they're deciding on it, so they need to know the state.

[00:21:41]And all and because it used to be we used to create the data once in operation store, and then we used to enrich it, integrate it, and then do analytics on it. Now we're just constantly creating more and more data that we don't even have control over. Huge cost implication.

[00:21:59]Tokens are expensive, but we'll come to that.

[00:22:03]All right. So, I'm going to move on to the analytical data stores. So, we all know how the importance of open table formats like Parquet, Parquet, Delta, Hoodie. There's another one called Paimon.

[00:22:19]And so so that has has actually transformed a lot of proprietary data stores. Like it used to be So, let me go to this unbundled computer engine. What do I mean by unbundled? When we got into cloud, we were very excited to see that storage and compute have been disaggregated.

[00:22:43]But we didn't learn the full story.

[00:22:45]Disaggregated is not same as unbundled.

[00:22:48]Snowflake may may say you don't have to scale your compute and storage together. We've disaggregated, but it still comes from Snowflake.

[00:22:58]But now with with uh, with Parquet, I don't need Snowflake uh, a compute.

[00:23:05]I could store it uh, in uh, Snowflake proprietary and then use any uh, any uh, Iceberg rest catalog computer engine or I can store it in in uh, Iceberg and then I can use Snowflake as a compute. I can use ClickHouse, I can use DuckDB, I can use Flink, Spark, Pandas. So, all of a sudden we come into this environment where we have completely not disaggregated but unbundled and now I have one copy of data. So, my goal in life is to have one copy of data and then have multiple use cases, multiple personas but not make copies of data.

[00:23:49]Uh, I we've all been in this industry long time and so we all remember how many we had data marts, silos of of of data and it was so hard to figure out which is the right source of truth for customer record because there are 10 copies of of that. So, we we now have an opportunity to not do that, have one copy but give end users the right analytical tool to run whatever queries they want.

[00:24:18]So, so what is more important than having a system of record now is the a system of context. So, so whether whether we believe in agents or AI or not, the whole idea is that when I run a query, I don't care where the data is sitting. That that storage of data, that concept has been abstracted away. We care about the context and why context is is really important is because when I said this customer service example, so this customer this uh, guy uh, on on the phone is answering the phone uh, inquiry from an irate customer. So, now you need to know everything about this customer. What did they buy? What are the product specifications?

[00:25:10]Why did they buy?

[00:25:12]That is not in the database. Why is never stored in the database.

[00:25:15]You see? So, just imagine this.

[00:25:17]What does a database store? It stores the transactions.

[00:25:20]It never stores why.

[00:25:22]So, if Dora buys a new laptop, why did she buy MacBook Air?

[00:25:28]Let's say. Do you have a MacBook Air?

[00:25:30]No. Okay.

[00:25:32]>> [laughter] >> I have a MacBook Air. So, what research did I do? That's not in Apple's database.

[00:25:38]So, so and why did I buy MacBook Air?

[00:25:41]Why not MacBook M5 Pro? So, all of this stuff is context.

[00:25:47]And if you have this context, then you certainly have so much more richer information. Snowflake bought, for example, a company called Observe, which is in observability space. Why do I need Why did they It's a data cloud company.

[00:26:02]Why do you need to buy a data dog type of observability company? That is because when developers work on on writing their their SQL code, let's say, or the application, application has, let's say, some failure. Applications are developing traces, logs, metrics, events.

[00:26:24]All of that is going to be in a different system.

[00:26:26]So, if I'm a developer and my application fails, you know what I have to do? I have to raise a ticket to the SRE. SRE hits me at that point because they have their day job to do. The SRE has to then go find out what why did what does the trace say about my application? All of a sudden, if I can bring it together, then I, as a developer, with a natural language assistant, can ask a question of why did my application fail? And it says, "Oh, according to the trace, you had an exception fault, and here is a problem with what you wrote." So, so the context becomes really important at this point.

[00:27:05]And then the final thing is is is that we are again in a yet another thing that is happening is 2 weeks ago when I was at Google Cloud Next, I was surprised to find out egress fees have gone away.

[00:27:21]How many of you knew egress fees are gone between cloud providers?

[00:27:25]So, okay. So, one hand. So, Google, AWS, Azure, they all have now fast cross-cloud interconnects. And the whole idea that we are getting into today is if network and cost is not a bad barrier, then leave data where it originates, and then federate it on the fly.

[00:27:48]So, now and by the way, all of them offer this. I could be in AWS SageMaker Lake House, and I want to see my SAP data. I can basically like I we'll talk more about this.

[00:28:00]There's a whole diagram I have. So, so this whole notion of move SAP data here and move Salesforce data there and then do all kinds of enrichment, all that is going away because of this fast interconnect between cloud providers with no egress fees and making using open standards like Delta Sharing to make the data discoverable in one place.

[00:28:27]So, so all of this has got nothing to do with AI, by the way. This is just This is where the analytical space is going.

[00:28:35]So, we've spoken quite a bit about operational and analytical, but what's in between? You need data engineering, right? You need somehow to connect this.

[00:28:45]So, in data engineering, uh I was working on my book, I I came up with this concept called ECL, which is in addition to ETL, and this is called extract, context, and link. And we'll talk about this. Uh how many of you have been looking at uh LinkedIn uh post on context graphs? Any hands? Context graphs?

[00:29:08]Okay, there's going to be a few of you.

[00:29:11]By the way, I I lead a very boring life.

[00:29:13]I get all my entertainment from LinkedIn seeing Snowflake and Databricks duke it out. So, So, context graphs actually broke LinkedIn this year.

[00:29:24]This is how like So, there is a uh VC company called Foundation Capital. So, two people from there, they came up with this idea of context graphs, and all of a sudden, uh everybody's been talking about context graphs. So, uh so, we and I I'll show you in the diagram. So, basically, the data engineering space has always uh been data originates in your data sources, operation systems, or or unstructured sometimes like tweets and all, and then you you move that data to analytical. In the process of moving that data, uh you have to build your pipelines.

[00:30:07]All of our structured data, by the way.

[00:30:08]Mostly mostly structured data.

[00:30:11]Now, what we are saying is that I've already established why am I moving my data? Because I have cross uh cloud interconnects, I have uh data formats, I can expose uh a lot of uh content. So, why in the world am I doing ETL? So, and if agents are going to do work on my behalf, they don't care about data can remain where it is. They care about the context. So, and the context is not just structured, it's all kinds of unstructured PDF documents. For example, I could say that I'm talking about unstructured data engineering. For example, I could ask questions about my company's S-1 filing. S-1 filing, by the way, is 300 pages. It is filled with diagrams, tables, and table on page 10 is related to to something on page 45. How does it know like how would you know these things? So, so unstructured data engineering is a very complicated thing because it's not there's no schema.

[00:31:16]It's very contextual. And and so now we see that there is a lot of movement in unstructured data. And then we talked about like federating data where it exists, zero copy.

[00:31:31]And in fact, you hear about zero ETL.

[00:31:33]How many of you have heard of zero ETL?

[00:31:36]Zero Okay, so quite a few of you. So, so that's the whole idea is Okay, so so what I did was I was thinking about like how do I depict it in a picture? So, if you look at the far left, you see data movement. This is what we do today. ETL, ELT.

[00:31:56]And then it says at the bottom real-time, real-time data movement. That could be using, you know, change data capture like Fivetran or Stream or Oracle GoldenGate and things like that.

[00:32:09]We could even be materialized view. I create a materialized view, I move the data to my target, and then I do a change data capture, and I keep it up to date. So, this is the the predominant technology-based data integration.

[00:32:25]Now, at the bottom is more of an infrastructure-based.

[00:32:30]And by infrastructure-based, what it says is let's not move the data, let's move the metadata. So, by moving the metadata across to the destination, I know where to find my source data and so now it's zero copy and it's federation and you can see delta sharing. I talked about that. On the right-hand side, this is brand new. This is where I'm saying that in the multi in the multi-structured structured semi-structured unstructured world, if I can extract the the context and then I link that context. I link it through some sort of a graph, a knowledge graph. In fact, this is where context graph is coming up. Then I get a complete picture of what's going on, not just who bought what, but why did they buy it?

[00:33:25]So, that linking so that's the extract context link, that linking or creating that context graph certainly makes my queries so much more intelligent. So, this is where I see the future is moving to.

[00:33:41]All right, so we're going to keep trudging along. So, I'm going to demystify what is an agent. So, let me give an example. Let's say I have a personal agent, Open Claw.

[00:33:55]Again, I'm going to ask how many of you know about Open Claw?

[00:33:59]Okay, most of the hands. So, Open Claw actually suddenly took AI by storm because it allowed you to create personal agent that would literally connect directly to my email, my calendar, read my calendar, and because it's based on on large language model, it can read my email and it can say, "Oh, wait, this email came from from my boss and my boss is very angry.

[00:34:29]And it's promotion time. So, I better act on it now. You see, it's I'm just making this up, but my point is that it could be from a customer, a prospect.

[00:34:38]So, an LLM can detect, an LLM can do sentiment analysis in just single shot.

[00:34:44]And and then it can So, that's a perceive. Then it can reason and say, "Okay, what do I need to do about this email that just came in? Maybe I have different plan of action." And then I'm going to calculate the cost, I'm going to calculate which what action to take, act upon it.

[00:35:04]Let's say my prospect says, "I'm not available to meet you on Tuesday next week, but I can meet you on Wednesday."

[00:35:11]Maybe it should do an MCP server call to United Airlines and rebook my flight to Philadelphia.

[00:35:20]So, so the thing like that. So, it's acting. And then the best part is it is the last piece, which is what we technically call recursive self-learning. It's learning. If it makes a mistake, it's uh, and I say, "No, you screwed up." It says, "Okay, uh, next time I know what not to do." So, it's constantly learning. So, so if you see on the headline, I made without human intervention red in color because a lot of times I go to an event, they say, "I built an agent. You ask an agent a question." And and look, it comes back and asks you for clarification, then you do this. I'm like, "That's not an agent. That's a chatbot. That's an assistant. An agent is autonomous.

[00:36:03]And an agent is uh, is stateful. What What do I mean by stateful?

[00:36:09]In the early days of AI, we would ask a question, even now, we go to ChatGPT, we ask a question, and ChatGPT gives us the answer. Now, without memory, although now it has memory, next time I ask a question, it is going to going to uh uh it won't remember what transpired before that. But, agents are stateful. They they know the previous action. That's why that operational database thing is really important because they have to maintain the context of state memory uh somehow so they can then keep uh being a data additive on top. They're calling tools.

[00:36:50]So, this whole idea that let's not build everything in an agent, let the agent call a tool. A tool could be that I'm saying that uh that I need to change my airline reservation. So, it's calling United Airlines MCP server, making that change.

[00:37:08]It's reading my calendar, it's reading my database, it's finding out who this client is, what is the address of this client, is maybe booking Uber for me uh as well. Well, that's all right. So, all of those are tools.

[00:37:21]And uh and it is uh it's not a sequel, which it means to the end. It is goal-oriented. So, that's what an agent is. It's It's autonomous.

[00:37:34]It has memory, it's stateful, it calls tools, it calls other agents. For all of these things, by the way, there are protocols. I keep talking about MCP, there's agent-to-agent protocol, there's a protocol agent payment agent pay protocol also from Google like agent-to-agent, which is uh it's Mother's Day is coming up.

[00:37:56]I'm on the road. It should understand what does my mother like, my wife in this case, um uh and you know, just go like uh make that that uh complete that that shopping, but we use open standards uh commerce uh protocol. So, all of this is happening.

[00:38:16]In fact, something new just came out on chat GPT uh 2 days ago. Uh uh GPT 5.5 instant where it even shows you memory explainability. It says, "I gave you this answer because you asked this, you did this, you did this." So, there's a whole panel that shows up in in ChatGPT.

[00:38:39]And then I can tune it. I can say, "Oh, I I want you to forget this memory of mine. Delete it or don't use it for this." But and so so this level of customization only got introduced this week. So So, that is what an agent is.

[00:38:54]So, I I hope this is providing a little bit more clarity on what's an agent versus what's an assistant.

[00:39:01]All right. So, now let's look at what is happening in the space of AI. So, retrieval augmented generation was the first killer approach that we used. So, what's rag?

[00:39:13]Rag basically is an ability for me to feed extra information into the large language model. LLMs are very big deal. They cost They need 100,000 GPUs to train and they train at a point in time.

[00:39:32]Everything that happens after they're trained is not in in in my LLM.

[00:39:38]Uh and in fact it's very interesting just I on a side note, there is a a funny meme that's going around.

[00:39:48]This is happening right now. You ask an LLM, "Um I want get my car washed. The car wash is 50 m away.

[00:39:57]Should I drive or walk?"

[00:40:00]And the LLM says, "Oh, it's 50 m. You should walk."

[00:40:03]So So So, anyway, but that's that's got nothing to do with it. So So, rag So, the point of rag is if I'm if I'm going to tell rag to summarize my document or or to predict uh churn rate or something. I need to give it some information. So, I give it So, I do retrieval. I I write a SQL that retrieves the data. I put it in the prompt and I send it over. It's stateless.

[00:40:32]Second time I I need to do the same thing, I have to do it again every single time. So, rag was just like In fact, when World Wide Web first came out, it was just static pages.

[00:40:43]And if I wanted it to look at database, there was this concept called CGI CGI bin as some of you remember, because they didn't know how to look up a database. So, then over time this got improved.

[00:40:58]If you don't want to do rag, then the other thing is you can fine-tune the model. You can take an open weights model like Meta's llama, although llama's fallen behind. The The The place where people go today to get open weights model is China.

[00:41:15]The all top Quan from Alibaba, Deep Seek you've all heard of, and whole bunch of them. These are models that are open weight. So, you can take you can download it. You can change the weights, and you can and you can say, "Here's my additional data."

[00:41:32]So, you can fine-tune it. You can train your own model. The problem is now you need a data scientist.

[00:41:39]And if your data scientist doesn't know how to do it, then you ruined the model.

[00:41:42]So, that's why this piece has not taken off.

[00:41:46]So, what has taken off is this stateful multi-agent orchestration.

[00:41:52]Orchestration is actually the is is a hot topic right now, where you have multiple agents, like I I've been talking about for the last few minutes.

[00:42:00]What is happening now are these uh uh built-in agents. So, role-based agents.

[00:42:08]And these agents understand what you do.

[00:42:11]They show up like for so uh so there is this concept that is very very popular and tropic introduce it called skills.

[00:42:21]What skills says is that everything that I do in my life at my job. So let's say I'm a database administrator. Every morning I come, I check to see how much storage space I have, security issues, this this this. I put everything down in English in markdown file and I give it to AI and say, "Take this and create me an agent." And then that agent will go and do all the things that you asked. It's multi agent, it will call tools, it will do whatever it needs to do, but that is the where we are now heading to. So completely self-service.

[00:43:03]And so so to do this, you need the environment. So it's not just the agent. You need memory, you need tools, you need some sandbox. It's really important to evaluate, make sure it's correct.

[00:43:22]That whole environment where you operate is called harness. And harness is where a lot of like LangChains of the world, they're all moving into the harness engineering, creating the harness. It's not just a model or the data, it's how you orchestrate between these.

[00:43:42]All right. So we are coming to towards the end. So I want to talk about governance and sovereignty.

[00:43:49]So what's happening in governance? So a lot of times I see people talk about I'm standing up a team for AI governance. I'm like, wait, a separate team for AI governance? We've just talked about like unifying everything into one stack, one use case, and now you're going to have a separate stack?

[00:44:08]So, AI governance is a whole beast by itself, but it should never be a separate stack. I tell people, this is the golden era of data because AI is a use case of of data.

[00:44:21]AI is sitting right on top of data. So, so why would you have a totally separate stack? So, unify it.

[00:44:28]Use data as the foundation and then build what what you need to do for AI.

[00:44:33]What do you need to do for AI? So, this is the funny thing.

[00:44:38]Data governance, how did we govern it?

[00:44:41]We govern the input.

[00:44:43]Like I I had my lineage, I had my data quality, I had my catalog, all of that.

[00:44:49]If I know where this data is coming from, and I and I make sure that data quality is high enough, I feed it to my application, and I'm done at that point in time. But in AI, even with the same data, same model, you can get different answers. So, all of a sudden, governance moves to the output because it's probabilistic.

[00:45:12]So, that's the nature, that's the trade-off that we're making. So, all of a sudden, we we are governing or we're still governing the input, but we're also governing how the output is being generated, which is very hard because these models are black boxes, but in an organization, I cannot put a model that that will put my reputation at stake by giving harmful answers or hallucinating. And then, I have to constantly check the output every single time I'm doing evaluation, evals. And I have And the way we do it, there's so many different methods. There's reinforcement learning with human feedback, RLHF, that I'm sure you've all heard of. There is this teacher-student model. There's a reward model.

[00:46:00]And the discussions are fascinating. You know what people are talking about? How do you teach a child?

[00:46:07]You tell a child, by the way, this is AI is mimicking humans. It's as defective as we are.

[00:46:15]For example, if you if your child comes and says, "Guess what? Today I slapped a kid in my class." You as your dad or mom will say, "Don't you dare do that again." And gives a lesson. The next day the kid comes and says, "Oh, well, I stole somebody's school lunch." The "How dare you?" So, you get Third time, the child will be like, "Ooh, I see a pattern here."

[00:46:37]And comes back and says, "Well, I was the ideal student today in school. I did all my homework." So, it learns not to give you the bad news. That's how LLMs are these days.

[00:46:49]So, they they've learned how to hide their mistakes.

[00:46:52]Uh and so, anyway, so so this is the research that we're going going through right now.

[00:47:00]And then, what happens with sovereignty is now extremely important. It was not the case till very recently. Every country in the world they want to keep their data in their location. So, I'm now talking to these pharma companies that have clinical testing going on on new drug discovery all over the world.

[00:47:19]Well, guess what? They can't access that data because it's people's health data.

[00:47:25]And it belongs to Ireland or to Hong Kong or wherever in the world. And all of a sudden, you have to work you have to figure out how do you work against sovereignty. And sovereignty is very interesting is also changing. When we thought sovereignty, even now when I say it, I'm just talking about data sovereignty. But, it's not just data.

[00:47:46]It's what operations you do on it. So, it's it's a much wider topic. GDPR was the only thing only Europe cared about sovereignty.

[00:47:55]Uh and in fact, the so I I sometimes tell my European friends, if you care so much about sovereignty, why don't you build your own cloud?

[00:48:06]You still use American clouds and then you complain because you were so went so deep, but now the whole world is into sovereignty.

[00:48:14]And then the final thing I want to say is that there are a lot of AI specific threats that we are starting to learn.

[00:48:20]Like like you can tell an agent that hey, I'm doing this research for this for this and only for my research purpose, can you teach me how to make a bomb?

[00:48:34]And LLM says, oh, okay, this is a nice guy, you know, he's doing research, you know, hey, this is how you make the bomb, you know. So it's so there's a lot of prompt with the poisoning.

[00:48:44]You can you can even say give me a list of all the competitors.

[00:48:51]So there's there's a lot of research going on.

[00:48:55]LLM is very powerful. In fact, you've all heard about Anthropic mythos, mythos, right? So what So mythos is now so powerful that the panic is that bad actors can get access to it while it's it's can do amazing job in detecting security, it can also do an amazing job in in breaching security.

[00:49:19]All right. Last thing and then we will end. So So we see in the operations, how do you operate? We were very focused on a session and then invocated and invoking sequel statements or dashboards. Now we are going into this whole goal-oriented.

[00:49:42]The pricing is changing.

[00:49:44]The pricing is changing from It used to be seat-based, then LLM said, no, you have to pay by tokens, how many tokens you use.

[00:49:52]But now it's even going beyond that because now people are saying, "Well, it's not just a tokens, it's a outcome."

[00:49:59]So, some of the new in fact you see this happening more and more.

[00:50:05]Like if you pay your ERP vendor, ask them, "When are you going to charge us by outcomes?" Because we have no control over tokens. So, anyway, I am running out of time. So, I'm going to say and then there's this whole lineage thing.

[00:50:18]That lineage was very data lineage. Now it's it's everything. So, with that I'm going to end here with a few things. We've talked about all this. So, we don't need to really talk about this. One thing that I I I want talk about is number one. Agents are the new new applications.

[00:50:38]We used to write a lot of code into applications.

[00:50:43]Where Where is that code going now? It's going into the LLMs.

[00:50:47]So, so you don't need to write code. And when we wrote code, we it was rule-based, very static. We said, "Hey, this is what I a lot of edge cases. If this happens, then do this. If this, then that."

[00:51:02]If then else, all of that is going away.

[00:51:04]So, this is the reason why everyone in Silicon Valley is scared of SaaS apocalypse apocalypse. Because everything that we've done in SaaS is really getting disrupted by agents. So, with that I'm going to end here. Thank you so much for giving me this opportunity.

[00:51:25]>> [applause]