Your AI agent still has no memory? Fix it with these 3 patterns

Added: 2026-05-15

[00:00:00]In our previous video, you showed us three ways of making agents smarter by adding memory to them. Yeah, we've talked about first session [music] state and second multi-agent state and last persistent memory. But you teased that there are three more ways of giving agents memory.

[00:00:18]>> Yeah, that's right. And let me show you.

[00:00:30]Welcome back to the show, Annie. Uh what's your story here at Google?

[00:00:34]Thanks, Martin. You know, I spent years as a software engineer here, but I realized that my real passion is AI education. So now I'm over on the developer side helping developers building AI agents.

[00:00:46]Yeah, you've been deep in the world of agents, right? Yeah, and I noticed the pattern. Everyone thinks AI intelligence is just about the model and tooling.

[00:00:57]They're forgetting the other important piece, that is memory. Guilty. I'm definitely focused on the model first when I'm building AI apps. Yeah, most people do. But if you want to build an app that actually impresses people, you need an agent that use a full brain. And here we have six patterns for adding agent memory.

[00:01:19]>> [snorts] >> And in our previous video, we talked about the first three. Yeah, that's right. And let's talk about the last three and when you should use them.

[00:01:28]I've also built a codelab for this so that you can try it out yourself.

[00:01:33]Okay, I will add a link to your codelab, Annie, in and our last video in the description below. Let's dive in.

[00:01:40]>> Sure.

[00:01:42]All right, let's talk about callbacks.

[00:01:45]You know, sometimes you need to update memory automatically based on the conversation. You want a spy that watches and takes notes. A spy? That sounds [clears throat] mysterious.

[00:01:57]What's an example? Oh, let's see. We want our trip planning app to nudge the user to new types of sites. And here, the user asked for a museum.

[00:02:07]So, if the user asked for another museum, the app suggests a different type of attraction instead. So, there's no duplicated activities that planned by the agent. I see. So, it's like having a dedicated assistant who's constantly updating their notebook while you talk.

[00:02:24]>> Yeah, that's right. And here is how to implement it. So, whenever an agent runs, you can add your own custom logic before or after the agent runs, before or after the model is called, and before or after any tools are called. And this callback intercepts the life cycle of the agent.

[00:02:46]And this is great for saving context about the conversation without making the agent itself more complex. Got it.

[00:02:53]And what does the code look like? Yeah, here is the after tool callback.

[00:02:59]So, here we set the activity type to cultural, food, or outdoor, depending on what tool was called.

[00:03:07]And here, uh we write that activity type to the tool context.

[00:03:13]And over here, in the planner instructions, we read the last activities.

[00:03:19]If it was cultural, the museum expert is banned.

[00:03:24]And the agent build its own context simply by doing its job. Nice. All right, let's talk about custom tools.

[00:03:33]You know, so far, memories have been unstructured text. It can be a lot of data for the agent to dig through.

[00:03:41]But if the agent needs to know that you're vegan, let's put that in a user profile. So, it's moving from unstructured chat logs to structured data. Yeah, exactly. So, if you tell the agent that you're vegan, it doesn't just remember that was said. It proactively updates a dietary preference field in your profile. Okay, and what would the user experience be here? Sure. So, here I'm asking the agent to plan dinner for me.

[00:04:13]It doesn't know my preferences, so it asks me.

[00:04:17]So, I tell the agent I'm vegan.

[00:04:21]And now I restart the script.

[00:04:24]Again, I asked it to plan dinner for me.

[00:04:27]And now it knows that I'm vegan, so it doesn't ask. Instead, it just suggests a vegan restaurant for me. That looks useful. How did you implement that, Annie? Sure. I gave it two new tools.

[00:04:40]The first tool is save user preference, and the second tool is recall user preferences.

[00:04:48]And in the agent instructions, I tell the agent to call recall user preferences first.

[00:04:55]And here I tell it to call save user preferences whenever the user states a preference.

[00:05:02]And then the save user preferences takes a dictionary object of preferences as an input parameter.

[00:05:09]And it then saves that dictionary as a new record in the database.

[00:05:14]So, this recall user preferences tool just read that record back. Very good. I can see how having structured data can come in handy. What's next on your list?

[00:05:26]Yeah. So, the last pattern is multimodal memory. You know, all our saved memories have been text so far, but humans remember more than text. Right. We remember the vibe of a photo or the sound of a voice or the feeling of a video, right? Yeah, exactly. So, a good agent should remember those things, too.

[00:05:48]And in this example, I've uploaded picture, a video, and an audio file with a travel assistant agent.

[00:05:57]And then I will ask, "Based on the picture, video, and audio I shared with you, where should I travel?" So the agent then reply based on my preferences.

[00:06:07]And it's connected the dots across different media types from the past.

[00:06:12]That's right. Here's a code that puts a session with a picture, video, or audio into the memory bank.

[00:06:20]And you can see here are memories in the memory bank.

[00:06:25]And here I give the agent the preload memory tool so that it can use to load a memory from the memory bank into the conversation. Excellent, Annie.

[00:06:35]I knew how to implement session memory before, but I haven't actually seen the code for the other patterns.

[00:06:42]Uh could you give us a quick recap, please? Sure, Martin. So firstly, we talked about how to use callbacks to update memory automatically without adding complexity to the agent. And secondly, we talked about how to use custom tools to read and write structured data like user profiles. And lastly, we talked about how to use multimodal memory to remember photos, videos, audios with what has AI agent engine memory bank. That's a great list.

[00:07:10]Uh thank you for sharing this with us, Annie. Yeah, thank you for having me, Martin. And thank you, everyone, for watching. If you have any questions for Annie or me, please let us know in the comments.

[00:07:21]Also, do let me know what you thought of today's episode.

[00:07:25]I read every single comment.

[00:07:28]Until next time.

[00:07:36]>> [music]