AI Agents That Remember: Building Stateful Systems with Lakebase

Added: 2026-05-23

[00:00:00]The challenge is [music] that most teams are building working memory through their LLM prompts. So, they store some basic data in a database and call it done, but without the four different types of memory working together, you're missing about 75% of what actually makes the agent intelligent. Your agent needs to pull um relevant episodic memories and enrich them with entity facts, [music] apply procedural knowledge, and synthesize working memory in milliseconds.

[00:00:26]>> [music] >> Hello everybody. My name is Savannah Longoria. I'm currently a senior developer advocate here at Databricks.

[00:00:34]Um I joined back in June as part of the Neon acquisition.

[00:00:38]Uh before that, I was a solutions engineer and I've been working with databases for the past 10 years. So, you can say that I really love databases.

[00:00:47]Um and today we're going to be talking about AI agents and specifically we're going to be talking about building stateful systems with Lakehouse. The first part of this talk is going to be a little bit more theory. Um but yeah, let's start with the big picture. So, AI is democratizing software development in unprecedented ways and we're seeing data scientists, analysts, and even some business users uh building applications that were previously uh the exclusive domain of engineering teams. And this shift is exciting, but it's also creating new challenges and developers are needing to create complex systems, agentic systems without necessarily having decades of infrastructure experience.

[00:01:30]They need databases that just work and that leads us to why we're all here today. So, let's trace the evolution of how we got here. Um we started with LLM chatbots. Then we added rag, which is retrieval augmented generation, where we started supplementing the LLM with non-parametric knowledge and now the model can access your company's documentation, your data. And this kind of represents the shift that we're in today.

[00:01:57]Um architectures can access um and process external dynamic knowledge in inference time.

[00:02:05]Um and AI agents is the next uh evolution.

[00:02:10]And LLMs use tool capabilities, um advanced reasoning and planning, and they also decide in what order. And today we're currently in the midst of a major shift um into agentic systems.

[00:02:26]We have system architectures consisting of multiple tools, AI agents, and different components. But not all AI agents are created equal, and I like to think of the agentic capability spectrum as a spectrum across four levels. We have level one, which is essentially an LLM with instruction that runs in a loop to get the desired result. So you can think of it as simple as like a chatbot trying to understand your question.

[00:02:53]Um controlled flows is level two.

[00:02:56]Um this is where most most production systems are today.

[00:03:00]And I see this a lot, so I help run the AI agentic program at Neon, and this is definitely where um level two and level three are where I'm seeing a lot of uh the customers that I talk to.

[00:03:13]Level three is essentially routing in specialized workflows, LLMs that classify requests, um and route them to purpose-built workflows.

[00:03:23]So they're making more context aware decisions about how work gets handled.

[00:03:29]And level four is autonomous agents. Um I like to like kind of think of something like Waymo, um which you know is coming to Seattle.

[00:03:40]Um and these are like LLMs that plan their own sequence of actions. Um they leverage tools independently, and this is kind of where we're at and where we're heading. Now that everyone is on the same page about the state of AI agents, uh let's talk about the fun stuff. At the core, there are four different components to an AI agent. We have perception, so that's how an agent understands its environment. Uh tools and actions, that's how the agent essentially decides what to do.

[00:04:09]And we have planning.

[00:04:11]Um or yeah, planning. And then memory is that's a big one that we're focusing on today. Uh memory is essentially what transforms a simple query system response system into something that can maintain context and learn from interactions and provide consistent personalized experiences. And the traditional pop problem is that LLMs are stateless by design. When you call LLM API, whether it's OpenAI or Anthropic or any other provider, it doesn't remember anything from previous calls. Each request is completely independent, and the model processes your input, generates output, and pretty much forgets everything without that persistence layer. This essentially creates a fundamental scaling problem in production. You typically run multiple replicas of your application across different ser- servers for reliability and performance.

[00:05:02]You can't scale stateless agents across servers in a meaningful way. And this isn't a limitation that you can vibe code your way out of, you know, you need external persistence.

[00:05:13]So, on the right here, we have um persistence, which is durable storage that survives API calls and server restarts. We also have relationships.

[00:05:23]So, this is connections between users, conversations.

[00:05:27]And memory is the like retrieval of storage and relevant context.

[00:05:33]So, if we think about what this means in practice, we have consistency across sessions. So, like I said, it requires external persistence, which that's why we're talking about LakeBase. Um and trustworthy interactions uh require a mem- memory management system. So, not only the persistence layer itself, but a memory memory management system.

[00:05:54]And um you also need accumulated expertise. So, it requires both persistent and management. You can't just store raw conversations. You're extracting learned preferences, tracking behavioral patterns, and building analogies knowledge of entities and relationships. And this essentially translates into like the user experience being a positive one. So, now that we understand the three-part formula for agent memory, let's break down the four different types of memory that production agents actually need. So, if I were to click into an architecture of a modern agent in 2026, this is essentially what you will see.

[00:06:31]Um memory and AI agent breaks down into short-term and long-term. You also have perception, which is essentially the environment, tools and actions, and planning like we um showed in the previous slide. So, now considering the previous diagram, if we take a step further and just focus on memory, we have four different distinct types of memory. We have working memory, so that's our LLM's context window. It's fast, limited, and temporary. We have episodic memory, which is the complete interaction history with semantic and search capabilities.

[00:07:04]We also have entity memory, so that's structured knowledge about people, products, relationships, and facts. Um this is kind of like some uh LLM knowing that the user 1 2 3 4 5 is vegetarian and prefers evening deliveries and orders extra napkins and yeah.

[00:07:25]Uh then we have procedural memory, so that's the learned patterns um and preferences over time.

[00:07:32]And the challenge is that most teams are building working memory through their LLM prompts.

[00:07:37]So, they store some basic data in a database and call it done. But without the four different types of memory working together, you're missing about 75% of what actually makes the agent intelligent. And these memory types need to work together seamlessly. Your agent needs to pull um relevant episodic memories and enrich them with entity facts, apply procedural knowledge, and synthesize working memory in milliseconds.

[00:08:03]So, if we look at this diagram, we can kind of visualize the complexity of all this you have to met um manage when you're building an AI agent. You have working memory, caches, knowledge base, conversation store, and they all need to work together. And that's kind of where we hit our first major problem.

[00:08:21]It's that AI agents have memory needs and um infrastructure needs that traditional databases were not built for.

[00:08:29]Traditional databases force you to cobble together um multiple management systems, manage synchronization between them, and build custom infrastructure just to support a basic agent functionality. You end up building a memory management system yourself across multiple databases. Um so, like what I what I've seen in the field is people have Postgres for structured data, Redis for caching, a separate vector database, object storage. And now you're managing essentially five different persistence layers, plus all the management uh memory management logic to orchestrate them.

[00:09:05]So, it's really hard to scale. And that's why most production agents remain stateless or have very limited memory, not because teams don't understand the value, but because infrastructure complexity has is become prohibitive.

[00:09:19]So, we talked about agent memory and why it's important in the context of managing persistence, but let's be specific about why OLAP and OLTP infrastructures uh fall short.

[00:09:31]So, traditionally, that there's a gap between your OLAP and OLTP systems where data has to cross the system and network boundaries to move one to another and require custom pipelines and configurations making it essentially pretty difficult.

[00:09:47]This is when teams end up building complex ETL pipelines between systems which itself introduces latency and consistency gaps that break agent memory. And the problem is that AI agents need both simultaneously. They need OLAP and they need OLTP. And traditional architectures required moving data between these silos is often through slow ETL pipelines.

[00:10:10]We have resources that are provisioned and have capacity limits.

[00:10:16]You can't predict agent workload patterns. One viral interaction could totally overwhelm your database.

[00:10:23]Your operational data data is siloed in flexible architectures which we covered in the last slide. And the data platform integration requires fragile ungoverned ETL.

[00:10:35]Moving data between systems is brittle and bypasses bypasses governance.

[00:10:41]Agentic developer workflows also create safe safety hazards. New developers building agents can accidentally corrupt production data and expose super sensitive information.

[00:10:54]And support for operational AI is bolted on. It's not built in when a lot of folks start their AI agents.

[00:11:01]Vector search and embeddings and AI specific features often feel like afterthoughts.

[00:11:07]And that is essentially the gap that LakeBase fills. So LakeBase serves as a operational and OLTP database which is necessary for operationalizing the data.

[00:11:19]Feeds back into the agents and displays it in applications so providing low latency query and writing.

[00:11:26]This essentially solves the external persistence part of our formula. So now let's actually look at how it enables sophisticated memory memory management systems. So, what is Lakehouse? With Databricks Lakehouse, transactional workloads are no longer a separate concern for analytics and AI. It is a new architecture for an OLTP database.

[00:11:48]Lakehouse brings managed serverless Postgres directly into the Databricks Lakehouse, making OLTP a first-class citizen along Delta Lake, ML Flow, and Databricks apps. And if there's anything you take away from this talk someone asks you what Lakehouse is, I want you to take this way. It is actually Postgres. It's not a proprietary fork, Postgres like, or Postgres compatible. It's good old Postgres that you can download from the internet and run on your laptop.

[00:12:19]However, there are some clever twists with Lakehouse that solve the exact problems that we talked about earlier today.

[00:12:27]So, our architecture disaggregates compute and storage. And what I mean by that is that the customer data set doesn't reside on the host that makes Postgres run.

[00:12:37]We have a clear separation between the host that runs Postgres and the software where the data set resides. This separation plus our serverless architecture is what enables features, which I'll touch on in the next slides and also when I do a live demo, that make it perfect for agent workloads. So, often times when I see folks launch, we see agent workloads burst.

[00:13:01]Like I mentioned, there's viral moments.

[00:13:03]But because Lakehouse utilizes separation of storage and compute, it's serverless by default. There's no sizing, patching, or scaling decisions.

[00:13:13]And auto scaling handles surges and scales down when it's idle.

[00:13:18]So, compute and storage that are decoupled also help you have cost control.

[00:13:24]You're not paying for what you don't use. And you can like right size to any stick steady state. You can set your cluster to whatever I think it's 0.5 up to 32 right now.

[00:13:36]Um and you can set that auto scaling range. You can also scale to zero um so you don't have to pay for what you use.

[00:13:43]And let's talk about branching. This is my favorite feature.

[00:13:48]Um so branching and development allows safe experimentation for AI agents.

[00:13:54]So it gives you instant zero copy isolation. You can create a complete copy of your production database including agent memory, conversation history, user data in seconds without duplicating storage.

[00:14:08]We also have a get style workflow for data. So this enables proper CI/CD workflows for your database. So you can create a feature branch, um test your agent changes against real data, and merge it back when you're ready. And if you need to test how your agent behaves um with different memory states, you can always spin up a branch, experiment, and then delete it when it's done. You don't duplicate storage. Um there's no complex cleanup, and this solves the data uh safety hazard that I was mentioning earlier.

[00:14:40]Developers can experiment freely without uh risking their production agent memory. And this is a little sneak peek into branching. We'll uh show it more in the demo. Um and we have recovery and backups which I'll also touch in the demo. So snapshots create instant backups on a schedule uh from when the database can be restored instantly.

[00:15:01]Uh we have an API that features create, delete, and restore for a snapshot and scheduling as well. We're going to be doing a demo on Casper's Kitchen which is essentially a ghost kitchen demo company that uses uh Data Bricks Lakebase and AI agents to opera- operationalize data and build internal uh tools. We're in the Data Bricks catalog, um and this is where all the Casper's Kitchen uh events are streamed.

[00:15:27]So, we can see here that we have um internal stakes support.

[00:15:32]And we're going to go over here to uh jobs or to serving endpoints.

[00:15:42]So, what we're seeing here is our endpoint. Um so, this processes incoming support requests and returns a response uh including the amount of the refund and credits to apply. It also generates a response to give.

[00:15:59]And we're going to go over to jobs and pipelines.

[00:16:03]And we're going to open up our agent stream.

[00:16:08]So, let's hope the demo gods are with me today.

[00:16:12]Um so, what we're seeing here is our jobs and our pipelines. Um we're going to open up here and see our notebook.

[00:16:22]And we could see our process and support request function. We have our OpenAI client, and we have our serving endpoints, which we showed in the last section.

[00:16:33]And if we go down here somewhere, we have our process support request. Um so, this is the actual request. Um you can see that the raw support request is here.

[00:16:48]And uh we can write the response of the agent and to support agent report table. All of this is in lakehouse. So, um here we have our support responses.

[00:17:02]We can go in here.

[00:17:05]We can see our sample data. You can see that we have our support ID request, our order ID, user ID, as well as the agent response.

[00:17:14]So, this is actual um JSON. So, this is in the lakehouse and now we want to get it into lakebase. So, we're going to set up a sync table.

[00:17:23]Um, and it's pretty straightforward. All you do is, uh, pick your destination, uh, your project. So, it's be that one and then your branch. So, you can utilize different branches.

[00:17:37]Um, there's different types of sync modes and you can create that. I've actually already created it and I'm a little too scared to do it during a live demo.

[00:17:46]So, let's actually go back in our lakebase instance and see that here.

[00:17:56]So, we have our different, uh, database projects here. I'm going to go to Casper's kitchen support. Uh, we can see that we have our compute is 1 CU. It's active right now. We can also see that we have different branches that aren't being utilized, which are idle.

[00:18:11]Um, you can set up in different regions and connecting is pretty straightforward. You can have your, uh, Postgres. We have different connection methods here.

[00:18:22]Um, and you can connect it to different roles.

[00:18:26]And you also get monitoring of your, um, database as well.

[00:18:32]But, let's go back here, um, to my tables. And we can actually see, if we open up the support, we can this is our, uh, support agent report sync table. And we can actually see that it's already in lakebase, which is almost instantly. And we have our application here. This is in our Databricks app, so you can go back to your compute and then you hit apps and we're in our support, uh, console that I actually live coded.

[00:19:02]And here we can see that there's state in the agent. So, we can see that we have, um, history. Um, we're opening the request from Cameron.

[00:19:12]Uh, we can see that the raw the raw support message. We can see that they needed help with the recent order. Um and the agent said, "We'll look into this."

[00:19:22]This isn't really a great response, and we can actually um use Lake Base and Lake House to regenerate that agent. So, we can actually advise it on how to improve its message, and it will take that into consideration and improve it.

[00:19:36]We also have um ways to reply, and you can type refund and all that. We can also, if we wanted to, essentially add a branch. So, I'm going to actually create a new branch and show you how easy it is to do. So, let's do sub test.

[00:19:53]We can uh choose when we want to automatically delete this branch, so you're not um paying for that extra cost. You can do uh including the data from a parent branch up to this moment, um as well as past data.

[00:20:10]And we get our own connection string for that branch. It basically is a live coding with cursor, and we can actually do utilize branching um with cursor. So, you can see that I gave it a prompt, um and it actually calls the API and um uses branching to create a new branch for new security measures.

[00:20:40]>> [music]