The First Tool Call: The Hack That Sparked Agentic AI

Added: 2026-05-10

[00:00:01][music] >> Hello everyone. Welcome back to my code to care video series. Today I'm going to do what I think is an interesting topic, which is taking through the very first tool call as part of an agentic workflow. So as a reminder, when we talk about agentic AI at least when I talk about it, there are three three characteristics of an agentic workflow. One is that there's multiple LLM calls.

[00:00:37]So this might be competing agents or a writing agent and a critiquing agent and then a finalizing agent or something like that, but it's some kind of multi-step workflow usually with multiple LLMs in the in the process, at least multiple calls.

[00:00:56]The second characteristic is tool calling.

[00:01:01]So instead of just the LLMs writing back they're actually calling APIs or updating databases or launching browsers or doing things. And that's called tool calling and we'll zero in on that today.

[00:01:17]And then the third characteristic is um a non-deterministic workflow.

[00:01:28]So most softwares kind of deterministic.

[00:01:30]You do A, B, then C and then if this then do D, otherwise do E kind of thing, but in agentic workflow you're letting the LLM decide what to do next based on, you know, the response from the prior tool call or based on something else that it's finding in the environment. So the workflow, the steps are really decided by the LLM. So that's what agentic workflow is all about. Now this tool calling was invented like a year and a half ago or so, so it's not too too old and it has a an interesting start to it, which I'll tell you about tell you about now.

[00:02:05]Let's take the question that you might ask an LLM.

[00:02:10]What is the current population of Boston?

[00:02:24]You may recall how LLMs used to answer this question.

[00:02:28]Uh they might say um well, I I know the population as of my training date last year, uh but I don't necessarily know the current population.

[00:02:39]So, LLMs were kind of um restricted from doing current news and current events and and uh and things like that by um by this training date uh problem. All their answers had to come since you from their training and from their weights.

[00:02:54]But, somebody came up with this very good idea, which is uh letting the LLM call tools. So, basically what they expanded the prompt and they said uh either answer the question that I'm passing you from the user, or if you'd like to do a tool call, then respond back in XML.

[00:03:13]And the XML should look like um I'll show you the full prompt in in a moment. I'll show you a screenshot of it. But, you're basically requesting a tool call like web search.

[00:03:27]Uh and I'm giving you one tool in this case. It's called web search. And if you'd like to do a web search, respond back in XML with what um the name of the tool and then the arguments to the tool.

[00:03:38]So, what would you like to search for?

[00:03:40]So, it might be population of Boston.

[00:03:44]And I will call the tool for you.

[00:03:47]I will get the results, and I'll send you back the results, and then you could try again to answer the question. And if you'd like another tool call, uh go ahead and and do that. Here are the tools that you have available to you.

[00:03:58]So, instead of the LLM writing a prose response back, it wrote XML back. Then you as an application parse that XML, ran your tool, took the answer, and gave it back to the LLM. So, this one little technique um basically allowed LLMs to sort of get around that training date problem and be able to answer very current questions. And so, this was kind of the invention of the or the very first tool call technique was was web search.

[00:04:33]So, let me now show you what this tool call looked like. And you can see you're telling the LLM you have access to the following tool, and then you describe the tool. In this case, it's just one tool, but eventually it would become multiple tools. If you'd like to use the tool, then respond back in XML. If you don't need to use the tool, then then then just answer the question. So, that was basically the first tool call. And let me kind of diagram out the flow now.

[00:05:01]Okay, the flow is this. You have your application that um that took the user question.

[00:05:07]Might be a chat chatbot type of thing.

[00:05:10]Um you called the LLM with that longer prompt. It has the user's question, and then it has a list of tools. In this case, the LLM would either try and answer the question if it thinks it's current enough, or it might say respond back in XML. I would like you to call this tool.

[00:05:31]So, let's do the scenario where it responds back with a tool call request.

[00:05:36]So, you would take that you'd recognize that it's XML. You would parse it, and then you would call the tool, which is web search in this case.

[00:05:47]Now, all of us know web search from going to google.com, but you can actually call it uh through a Google API, as well. So, imagine there's not a user interface, you're just calling Google and then getting a text file back of the results. So, you take those results back from your Google call.

[00:06:06]And then you send them back to the LLM, just like a chatbot history.

[00:06:13]Like you're Like you're doing the search and you're sending the Google search uh results back to the LLM. And the LLM might want to do another tool call.

[00:06:23]Maybe the search results weren't good enough, or maybe they want to ask for the direct population versus the kind of wider city population or or something like that.

[00:06:33]Uh so, at any rate, the LLM might request more or might just answer the question, and then you'll recognize when you get the answer back that it's not XML, and you'll present that answer to the user. So, um so, that was the very first uh very first tool call was to solve that that problem. Now, tool calling has evolved. This whole thing is only like a year and a half uh old, but tool calling has evolved a little bit.

[00:06:56]So, phase one was this, where we were just asking the LLM to write XML and you were doing all the work associated with packaging up your tools and asking for results.

[00:07:10]Phase two is there were these frameworks, um the most famous is something called LangChain frameworks that did the XML for you. So, the framework you'd say, "Here are my tools." And the framework would add the XML request uh when it got uh a response back from the LLM, it would parse it up for you, might even call the tool uh for you, that kind of thing. So, it kind of took care of this um you know, the messiness of the XML. And then uh phase three, which is kind of the phase that we're in, that OpenAI and Anthropic added tool calling to their APIs. So now when you call an LLM through an API, you can say here are my tools.

[00:07:55]Here's the format that I would like.

[00:07:57]Here's the arguments for my tools.

[00:07:58]Here's how to use them.

[00:08:00]And then the APIs kind of take it take it from there.

[00:08:04]So that's basically how it got added. And then these companies have trained their LLMs to be very good at tool calling.

[00:08:12]So so that's been one of the training ambitions the last last few years is is to improve the tool calling ability of these models.

[00:08:22]And then the so far this is still very non-user-oriented, very programmer-oriented. But phase four, which is what we'll soon encounter, is that user picks tools.

[00:08:38]So as you use an AI system, it'll give you a list of tools and you could just pick those those tools and then the rest will be taken care of for you. So in these cases you have to write the tools yourself. In that last case, you'll have a grab bag of tools that you could pick from and your AI system would be able to draw upon those tools in terms of getting your your tasks done.

[00:09:02]So I hope so that's that's the quick history of tool calling. It's not a very old old history, but but I think it has an interesting start. It was a great idea to ask the LLM to produce XML instead of an answer. And then it just opened up a whole world of being able to get more things done, being able to be current with LLM answers.

[00:09:24]And now we kind of have this gigantic AI explosion that's partially based on this this very good idea.

[00:09:31]So I hope that was interesting. Until next time. Bye.