Tracing Agent Sessions with OpenTelemetry & Aspire

Added: 2026-05-19

[00:00:00]Hi everybody. I'm Maddie Montequila, and today I'm going to show you how to peek behind the curtain of what your AI agent is actually doing in VS Code using Open Telemetry and Aspire. AI coding agents do all sorts of work in our repos now.

[00:00:14]They edit files, they run commands, they call tools, but it feels like you hit enter, you kind of just follow along the chat, and stuff happens. But, there are ways to see what's actually happening behind the scenes, not just what it's doing, but what context it has around the task it's completing.

[00:00:32]We're going to use two things for that.

[00:00:33]One is Open Telemetry, or Otel, which is the industry standard for telemetry, like traces, metrics, and logs. Those are the ones you usually hear about. Um, and Otel works with pretty much any programming language or type of app or container or resource, so like a database or a cache. And because agents have become their own type of distributed app, what you type in the context it adds in the response and how it presents it to you is an end-to-end flow through resources, right? Otel is actually a really good way to trace, to monitor what is going on while your agents work, and and give us some visibility into the behind the scenes there. The other thing we're going to use is Aspire, which is a free open-source tool for developing and deploying apps, modern distributed apps.

[00:01:17]And like Otel, it also works across all different types of languages and resources. Aspire ships with an Otel developer dashboard. So, while you're writing and debugging your code, you can see logs, metrics, and traces in real time as if it was running in production, but on your like local dev machine. All you need to start with Aspire is the CLI, so it's super lightweight, and then you can use as much or as little of Aspire's functionality as you want. What we're going to do today is use it to start up the dashboard, and then have VS Code agent sessions emit some Otel to it, so we can follow along with our agent's work. So, let's take a look. So, I am here in VS Code, and we're actually in the Aspire repo. We're going to have the agent do some feature work for us.

[00:02:00]The first thing I need to do is start up the Aspire dashboard. So, using the Aspire CLI, I just do Aspire dashboard run, and then allow anonymous means that I don't have to configure any dev certs or anything for the dashboard to be able to talk to whatever is emitting the O tail.

[00:02:17]So, it spins it up on 1888 for me. Let's copy paste that. And then it exposes two ports for either GRPC or HTTP O tail metric. In the S code, to tell it to start emitting O tail, you have to do two settings. And they're both under GitHub Copilot chat O tail here. The first one is just enabling it. This is off by default, so make sure you check this box or you set it in the JSON, which enables the open telemetry trace, metric, and log emission for Copilot chat. And then the other one is capturing content. And capture content means it can actually see system instructions, input output messages, and tool definitions in the O tail. So, once you check those off, you'll reload the VS code window. The nice thing is you don't have to worry about the endpoint or anything because Aspire has the same default as VS code. So, 4318 for the HTTP app.

[00:03:07]O tail. Let's figure out we're going to do a bit of a meta issue we're going to ask the agent to look at today. It is about adding a VS code command palette entry to the Aspire extension to open the Aspire dashboard. So, a little bit of like Aspire VS code inception here, but we'll ask it implement a solution to, and then I'll paste on it. Yeah.

[00:03:32]We'll see. Let me copy this URL, close the terminal here, and open the integrated browser. Paste that in, and I can see this is the Aspire dashboard. It is being populated with a bunch of different spans. In this case, this is a structured log page, so I can see which model it's using. um, have auto mode on, so I can see it switched at one point between 4.0 mini to 5.3 codex. It used Gemini there for a sec. I can see the turns it's taking. Let me close the explorer, give us a little bit more room. And then I can go into any associated trace or the full traces page, which show what's actually happening across the entire flow of my agent's work. So we'll click on this one. It has eight eight traces here.

[00:04:17]And I can see it tried to it invoked GitHub Copilot Chat. It used Gemini. It tried to do a grep. It had some issue maybe it couldn't find the path, um, and all of these different things.

[00:04:29]Anytime that these sparkles show up, that means there's gen AI specific hotel related to this trace. So I can click on them and see details about the input and output, the tools it calls, any system messages or user context that is set. So in this case it's you're an AI coding research assistant. It's trying to figure out which which you know, issue it needs to fix.

[00:04:54]Um, I can dive into these.

[00:04:56]And then my user context, so it's gives it the working directory. It's searching across things. I probably should have told it it was a GitHub issue and not just a random number, but that's okay. Um, and then the tool calls it uses.

[00:05:12]And I can do this for literally anything that has these sparkles in my dashboard here. I can also go into metrics if I want to see specific like performance, number of things it calls.

[00:05:25]Um, you can click through all of these.

[00:05:26]They're super fascinating. Sometimes I like to just click around and look, but uh, if you if you have some specific questions, this is a really good way to get them answered.

[00:05:35]What's also cool about this is if I pop back open my terminal and do this, open a new terminal tab, I can ask the CLI, the Aspire CLI, to give me that same data in terminal format so I can do Aspire OTEL logs. It knows exactly what I want, and pass it the URL that this this dashboard is running on.

[00:06:00]And enter. Um and get the text version of everything I just saw. So that means you can use the Copilot CLI or the Copilot chat window in VS Code agent sessions or whatever other agent thing that emits OTEL for you, to use the Aspire CLI to look at its own traces and logs and do some sort of activity or inference or learning on it. Um the practical use case I've had for this is particularly like when I want to make sure that my Copilot agent instructions aren't too aggressive, aren't filling up too much system context, or or aren't giving enough context. Um and so I've used that a lot to like refine my Aspire repo instructions.

[00:06:46]This is a very, very big repo. Let's look at a fully Aspire-ified app that's a little bit more um friendly and give you a glimpse of what happens if you buy all in on Aspire, too. So this is an app I built that's just uh it's like three different APIs for three different transit systems: BART, New York's, and and the T in Boston. And I built them in different languages. So I have a Python API for the T, I have a C# API for New York, and I have a Go API for BART. I also have a what I'm calling the API the advisor, which looks at all of the service warnings across the transit system and gives me like a recommendation. Very cool.

[00:07:28]It's running right now uh on this localhost URL.

[00:07:31]Let's open the integrated browser here.

[00:07:34]I'll post back in the uh URL from the chat agent and then I will open a new tab and I will paste in this URL.

[00:07:45]Wonderful.

[00:07:46]And now I have two different Aspire dashboards running. One that's telling me what's going on with my Copilot chat and my AI agent session in VS Code and one that's actually observing my app itself. So, I see all of my different resources here that are defined in that apphost.typescrip t file.

[00:08:04]That's how you define things in in Aspire. It's just in in code in TypeScript or C#. And those get translated to resources in the dashboard. And then I get all of the ports and everything for it. I can click on my front end, go to the New York Subway cuz that's the one that I have the API key for.

[00:08:20]Click on different things. Da da da da da da whatever.

[00:08:24]Um and then go back into the dashboard and have the same structured logs and traces populated. So, we can look here and see it called my Redis cache. And that's all that happened there. Pretty easy.

[00:08:39]So, now I can use my agent to edit my app, get data from my app, figure out what's going on and any issues with the traces, and then I can watch my agent do all that through the dashboard as well. Uh and and use it to improve my agentic development or just because I think it's cool and fascinating to see what's going on. So, if you want to try this yourself, check out the VS Code tracing docs, the Aspire dashboard docs. All you need, like I said, is the Aspire CLI. And to set those two things in VS Code, it's super easy. Uh and if you want to join me and the Aspire team to learn more about what Aspire does, join the aspire.dev YouTube most Fridays. So, see you soon. Bye.