Behind the Cape - Context is the New Code: AI Engineering with Snowflake Cortex

[00:00:00]Hey everybody, welcome to another episode of Behind the Cape. I'm Data Superhero Keith Belanger. I'm excited to have fellow Data Superhero Karthik joining me today. Some of you may have seen him on some of our past episodes, but always a pleasure to have you back again. So, welcome Karthik.

[00:00:18]>> Uh thank you, Keith. Thank thanks for having me here. It's always a pleasure to do yet another episode with you.

[00:00:24]Looking forward.

[00:00:26]>> Yeah. So, for those of you who have not met Karthik, can you give them a little bit of background about who you are and a little you know, tell everybody a little bit about yourself.

[00:00:36]>> Okay, I'll start with a quick introduction for the benefit of the audience. Hello everyone, I am Karthik Raman. I'm based in India and I head the Snowflake practice at LTM.

[00:00:47]In addition, I also lead the Chennai Snowflake Chennai user group chapter and I'm also a technical blogger. So, what you see on the screen is like the my LinkedIn profile. So, feel free to connect with me. And always happy to share my knowledge and expertise with the larger Snowflake community and learn from them as well.

[00:01:09]>> Great. So, what topic do you have in store for us today for everybody?

[00:01:15]>> Uh this is a interesting topic like this is a theater session I did at the year summit like last week.

[00:01:24]Uh context is the new code. So, how we can do a context engineering with Snowflake.

[00:01:29]So, that is what we are going to cover.

[00:01:32]So, can we share the slides with the audience? Uh Is it okay?

[00:01:38]>> Yeah, I'm sharing them right now. You're good to go. Go for it.

[00:01:42]>> Okay.

[00:01:43]So, I want to [clears throat] start with a simple question. Many of us will have seen a AI demo that looks absolutely magical and later then like when you expose it to a real enterprise data, it might fall apart.

[00:01:56]So, I've been in the data and analytics space for more than like 23 years and here is what I've learned after spending years helping organization modernize their data platforms.

[00:02:07]AI does not fail because models are not intelligent enough. AI fails because it lacks context and that brings us to the central idea of today's session, context is the new code.

[00:02:18]Let me explain why.

[00:02:20]Uh let let's travel back of just a few years. We have spent like enormous amount of time enterprises would have spent enormous amount of time building pipelines etc. We wrote ETL jobs, we created transformation logic, we built dashboards.

[00:02:34]Every piece of intelligence in the system had to be explicitly coded by engineers. That was before the before Snowflake AI came into the existence.

[00:02:44]If the business changed, we had to change the code. If schema changed, we had to change the code. If report changed, we have to change the code.

[00:02:52]And then suddenly something interesting happened. The models became extraordinarily capable.

[00:02:58]And Snowflake Cortex arrived. Suddenly we could do things like summarize, extract, classify, search, reason without writing any more additional code, but just by providing better context. Now we are entering the context native era. Agents autonomously decide like what to do, when to do, how to do, what tools to use, how to change them chain them together. All that agents decide on autonomously. They agents don't just execute instructions.

[00:03:29]They decide, they plan, they orchestrate, they collaborate, which means the most valuable asset is no longer your code base. It is the context layer.

[00:03:39]That is the paradigm shift we are talking about.

[00:03:45]Moving on.

[00:03:46]Yeah.

[00:03:47]>> I was just going to say I can remember the good old days of having to write all that code.

[00:03:51]>> [laughter] >> Yeah, we all went through that and then like now I'm glad like we don't have to go through that like anymore.

[00:03:58]So, now that like we we have we accept that like context is the new fuel for intelligence, a far more important questions emerges.

[00:04:07]Where does that context come from?

[00:04:10]When we look at like large enterprises, context is not sitting in a single place. It is fragmented across departments, applications, policies, documents, metrics, and across across people. So, the moment you introduce multiple agents into the mix, you discover that agents intelligence is no longer the biggest challenge.

[00:04:33]Bringing in a shared understanding is the most complex challenge. In fact, if you look at the large enterprises, um the hardest problem is like not getting the agents to think, it is getting the agents to agree on a common shared reality.

[00:04:48]That is where like most agentic systems fail. When we are not struggling with the model intelligence like the all the models are like super brilliant like GPT-4, Claude, Llama, you name it, they are all brilliant. The struggle is giving the right context to the models at the right time with the right amount of guardrails. So, uh think about a scenario like what happens like when you ask a business question uh without context. The model will hallucinate, it will just like make up the numbers, and the model will give a confidently wrong answer. So, that is not a intelligence problem, it is a context problem. And what makes it even harder is context is not static. It changes with like who is asking, what is being asked, what role they have, what time of the data it is, when was the data last refreshed. So, it depends on a variety of factors. So, you need a dynamic, governed, real-time context delivery.

[00:05:44]That is where like Snowflake is a game-changer.

[00:05:47]It is one platform, one common semantic understanding, one governed source of truth.

[00:05:52]So that every agent agrees on the same common reality before taking an action.

[00:05:58]So um Snowflake changes like all the traditional problems like we had with context. Snowflake is a game changer there.

[00:06:11]Okay.

[00:06:11]Now moving on to the next part, okay.

[00:06:15]Now that we are we are looking at a fascinating question. If the context has become the foundation of intelligence, should we still thinking about the AI system the same way we traditionally thought about the old data engineering platforms?

[00:06:34]For decades, we always believed code creates behavior. But today, I would rather argue context is what like is going to create behavior. In traditional software, we wrote like hundreds of business rules. If this happens, do this. If that happens, do something else.

[00:06:51]We were not just writing if-then-else branches. We were basically passing data and instructions. And the basically the model used to derive behavior from that. The quality of what we pass as an input largely determines the output.

[00:07:08]So basically the model determines the behavior. The second thing is like traditionally we code queries the data. And like in the AI engineering on Snowflake, context is what like makes the data queryable.

[00:07:22]Business users like they don't want tables. They want answers. So semantic model is what acts as a bridge between the business language and the data. It describes what your data means, the metrics, the relationships, the filters.

[00:07:36]So the description is the context that makes natural language queries possible.

[00:07:41]And the third quadrant if you look at it in traditional world, we used to write a lot of code to connect systems, but whereas in the AI engineering world, we give context to do orchestrate tools.

[00:07:54]Agent node don't need like pre-defined workflows. They look at the context, the user question, and the available tools, the intermediate results, and based on that agents decide what to do, when to do, and how to do. Basically, context is the program.

[00:08:08]And the context becomes the workflow, basically. And finally, in traditional world, we used to write a lot of code to enforce like rules and all that, but in the AI engineering world, context is what like inherits the governance. Governance should not be after thought.

[00:08:25]That is the most important shift I feel.

[00:08:27]You don't in the AI engineering on Snowflake in the new world like you don't write separate the security logic for your AI layer. The Snowflake security policies that protects the SQL queries automatically protects the AI response once as well. So, basically, governance will travels with the context. So, that is the most important shift I would say.

[00:08:50]>> You know, one thing I want to just add here to what you're saying, Karthik, when it comes to context is the importance of getting to talk and communicate with your business cuz to right to get that context right, I've seen, you know, engineers and and folks assume the context, you know, without talking to their business participants and I think that's one key aspect to try to truly get that right context, you have to interact and talk with the people who will be using, you know, those those natural language queries and speaking in that context.

[00:09:29]So, you have to get that language down, you know, and put that into that into the context into the semantic layer, for example.

[00:09:37]>> Absolutely Keith, like you nailed it. The The business users like who knows the uh the functional domain, they have they have all the knowledge and then like that needs to be like clearly captured in the semantic model. That is absolute must.

[00:09:53]Moving on, there are like multiple different dimensions of context that I wanted to cover here.

[00:09:58]For me, I would I would broadly classify it as a five different dimensions of context. First is the business context.

[00:10:06]What does the data mean? When somebody asks a business question, what is the churn rate, for example? The AI needs to know what what is the table like that it needs to look at, what date filter, what formula. So, basically semantic model encodes all of this.

[00:10:23]Without business context, AI will generate a SQL that looks right, but it will do a wrong calculation.

[00:10:31]So, that is the difference like semantic model is going to make. The second part is the technical context.

[00:10:37]The technical context is more about how the data is structured. The column types, be it the joins, the partition keys, the freshness timestamps. So, all that technical context information is used by the agent to decide what Okay, what is the right approach to retrieve the data? So, should it do a table scan or full table scan or should it use a materialist view?

[00:11:00]All that decision is made by the agents using the technical context. The third part is the trust context. This is basically what controls what AI is allowed to see. And honestly, most AIs AI solutions fall apart because of the trust context.

[00:11:16]Snowflake security features like they define the boundary.

[00:11:20]Uh PII data gets masked even before it reaches the model. That is the difference between the good successful AI solutions versus the ones that fail.

[00:11:31]The fourth one the operational context.

[00:11:34]A lot of things are there like be it query latency, warehouse load, pipeline failures, task history, all that are the operational context. Imagine you're creating an agent that notices the morning ETL job run failed and proactively alerts downstream consumers.

[00:11:51]That is the operational context powering autonomous behavior. And finally, the domain context. What does the industry knowledge means? Uh You from the context point of view, this is the industry specific knowledge whether it's health care regulation, finance compliance rules, manufacturing shops, all that constitute the domain context. So, all these five context dimensions are equally important. So, here is a challenge like I wanted to leave the audience with. If they are currently doing any AI initiative, if they are struggling, it is not because the model is not performing. It is because one of these dimensions might have been like missed out. So, audience can can like look at their current AI initiatives and then like they can see if they are like capturing all the different five dimensions of the context that we I just spoke about. That is equally important.

[00:12:53]I'll quickly move on, okay? Now, now that we looked at the all the five different um uh uh aspects of context. Now now I wanted to talk about, okay, how do we do AI engineering at scale on Snowflake?

[00:13:11]So, now that we have established AI outcomes depend upon the five dimensions of the context, the next question is how do we operation operationalize these dimensions at enterprise scale?

[00:13:23]Most AI architectures today are like assembled from multiple products. There will be a vector database for retrieval, there will be a orchestration framework for agents, there will be LLM platform for inference, there will be a separate governance tool for security, and there will be a lot of custom code to make everything work together.

[00:13:41]But, Snowflake takes a completely different approach. It bring It brings the context engineering natively into the AI data cloud. So, there are four pillars I want to talk about. The first one is the Cortex Analyst.

[00:13:55]Cortex Analyst use the YAML-based semantic model to encode business meanings. These semantic models defines entities, metrics, dimensions, joins, aggregations, business rules, etc. When the user is asking a business question like in a natural language, Analyst does not simply uh Cortex Analyst does not simply prompt a LLM and hope for the best. It grounds the generation process using the semantic model, produces verified SQL against Snowflake tables. The important part is semantic model becomes a reusable context layer, exactly. Instead of teaching every application what to do, when to do, we define it like once and then reuse it everywhere across the enterprise. That is the first pillar.

[00:14:40]The second pillar is the Cortex Search.

[00:14:43]Enterprise context like doesn't necessarily live in like structured tables alone. It is lives across the unstructured data, whether it could be the PDF, contract documents, policy documents, knowledge bases, support tickets, etc., etc. So, Cortex Search provides a hybrid retrieval by com- combining vector similarity vector similarity search with traditional keyword retrieval. At runtime, uh documents are automatically chunked, embedded, and indexed directly within Snowflake, and relevant context is dynamically assembled at runtime and injected into the model. This gives a governed rag without introducing a external uh vector database. So, that is about the Cortex Search. The third pillar is the Cortex Agent.

[00:15:28]This is where the context becomes actionable.

[00:15:32]Agents can orchestrate multiple tools in a single workflow. They can invoke Cortex Analyst to generate SQL, use Cortex Search to query unstructured data, execute queries, interpret results, and determine the next action step of action autonomously. What is important is context flows across every step of the workflow.

[00:15:55]The fourth pillar is the Horizon Governance.

[00:16:00]Context without governance becomes a liability for any enterprise.

[00:16:04]So, Horizon governance ensures whether it is row access policies, masking policies, classification tags, lineage, RBAC controls. Together, everything every Cortex capability inherits these controls automatically. The model sees the data that the user is only authorized to access. And finally, Cortex Code, it brings all these capabilities together.

[00:16:30]It understands business semantics through Analyst, retrieve knowledge through Cortex Search, uses agents for orchestration, operates within the Horizon Governance framework, which means code generation itself becomes very context aware. That is the power of Cortex Code. What do you think, Keith?

[00:16:49]>> Yeah, I love the way you showed this because I think there are a lot of people that think of like Analyst Search and these agents as these independent things that can only use in singularity, right? And where you're showing it here is like, no, a given, especially with Snowflake Intelligence, a given natural language question might require that you're it's using Analyst Search, you know, and Horizon all at the same time, right? To bring back the right um answer to that question. It might have to scan, you know, um unstructured data and then it has to understand the context and then is that is that PII or sensitive data from a governance perspective so that answer comes back in the you know, securely, safely, and in the right context. So, I really like the way you show that here.

[00:17:39]>> Yeah, that is the power of Koko. It brings all these capabilities together nicely.

[00:17:46]>> Yeah.

[00:17:47]>> Moving on, I wanted to talk about um what one of the I want to before we talk about the Snowflake architecture, one of the biggest challenges with enterprise AI is the architectural sprawl.

[00:17:58]A separate model platform, separate vector database, separate orchestration frameworks, separate governance layer, all that. Each of these additional components in increases the complexity, latency, and mainly the operational overhead.

[00:18:12]What makes Snowflake completely different is all these capabilities are brought together on a single platform.

[00:18:18]Your structured and unstructured data, foundation models natively available within Snowflake, and then semantic understanding, agentic workflows, governance, all operating within the same environment. That means we are going to spend less time on like integrating the code, and etc. Fewer data movement challenges. And a single consistent security model. And a significant um faster path from proof of concept to production. So, enterprise AI becomes dramatically easier when the data, intelligence, and governance all exist like within the same unified platform.

[00:19:03]That is the power of Snowflake we are talking about.

[00:19:06]Um let's go one level deeper into Cortex itself. Every capability shows shown here contributes into a different form of context. When we look at the Cortex AI functions.

[00:19:19]Cortex AI functions like provide manage LLM inference directly within Snowflake.

[00:19:24]We can invoke models such as Claude, Lama, Mistral using SQL. The critical point is data stays within Snowflake secure perimeter. Inference always happens where the data resides, which is within the Snowflake platform.

[00:19:38]Cortex analyst contributes to the business context, I would say. Natural language is translated into SQL using semantic models. The semantic layer constrains the search base and improves um the accuracy.

[00:19:54]Without semantic grounding, text to SQL becomes unreliable, especially at large enterprise scale.

[00:20:01]The third one is the context search. It contributes to the knowledge context, I would say. It combines vector retrieval and keyword keyword retrieval. It provides automatic chunking chunking and embedding and delivers sub-second retrieval performance for RAG applications.

[00:20:17]Cortex agents, it contributes to the execution context, I would say. They maintain conversational memory, select tools dynamically, plan multi-step workflows, and orchestrate complex reasoning chains. Think of them as a intelligent control plane operating over your data ecosystem. And then finally, the Cortex fine-tuning. It contributes to the domain context. When organization can adopt like all the foundational models using proprietary data, with each training, which is done like within Snowflake, the result the resulting model becomes part of the same govern runtime. So, the way I would look at this, analyst provide business context, search provides knowledge context, fine-tuning provides domain context, agents provides execution context.

[00:21:07]And the AI functions provides reasoning.

[00:21:10]Together they create a complete runtime, which is the context runtime, which is the context Snowflake context.

[00:21:19]And we are at the end of the presentation. In conclusion, what what I wanted to say is the shift from code to context is not coming, it is already here. Snowflake is the only platform where your data, your models, your retrieval, your agents, your governance all live together.

[00:21:38]So I the audience, if you want to do one thing after the session, go and invest in your semantic models. That is the highest leverage move that you can do.

[00:21:47]It It turns your data platform into something AI can actually reason over.

[00:21:51]And remember, governance is not a nice to have, it is a must have. It is the thing that lets you ship your AI into production rather than being at a demoing stage.

[00:22:02]So with that I can conclude the presentation. So looking forward to have the audience like reach out to me on LinkedIn and then like happy to discuss if they have any questions.

[00:22:14]>> Yeah, I'm going to bring us back to the screen and that was great. You know what I what I find interesting Karthik is that what you talked about in context in reality from a data perspective. I mean, I've been in data for 30 years.

[00:22:27]It's always been important. We kind of lost sight of that over the years, right? We're always like, you know, let's just get it in the data lake.

[00:22:36]Let's just let data scientists at it.

[00:22:38]Let's just And the context was in the person's mind who was trying to use the data, right? We kind of shifted that the user needs to know, right? And their systems didn't need to know. But AI is kind of remind from my perspective, AI has reminded us about the importance of that because AI doesn't have the experience, doesn't have, you know, the contacts and we have to now reintroduce what that into our development processes and in order to get it right. So, it's kind of like reversion is history. I guess you could say it's like something that we had to do many years ago.

[00:23:18]And and now we kind of got away from it, but it's really in critical importance.

[00:23:23]So, I thought I know I really enjoyed your your presentation and it's kind of a good reminder for the listeners that are out there of the importance of that and putting that into your development life cycle and building those relationships with your customer again and getting to understand the business. So, thanks for that. That was very well that was that was great.

[00:23:43]>> Thank you, Keith. Thanks for the opportunity. Context again is going to be the most important thing in any AI systems that we are building and the customers are more and more spending a lot of time investing in building the right context for the AI tools and solutions that that we are building. So, it was a great opportunity for to be collaborate with you one more time and then share my experiences on this. So, looking forward to doing many more such sessions for our Snowflake community.

[00:24:13]>> Well, thanks. Thanks for coming and and joining us today, Karthik. For the for everybody who's listening, you should be able to find his link to his LinkedIn below in the comments and until our next episode, thanks everybody. We'll see you all later. Bye.

[00:24:28]>> Thank you.

[00:24:36]>> [music]