Full-Stack Kotlin AI: Powering Compose Multiplatform apps with Koog & MCP | John O'Reilly

[00:00:00]Um, hi folks. Um, my name is John O'Reilly. I'm based in Galway in the west of Ireland.

[00:00:07]I work remotely for a Norwegian company called Neat and it's an absolute pleasure to be here at Kotlin Conf in Munich today.

[00:00:14]So, we're going to start today by giving a brief overview of some key concepts and we'll dive in then to a live coding session where we build out a Compose Multiplatform application that's powered by a Kook AI agent.

[00:00:25]We'll show how we can use local tools to provide particular data needed by the agent and we'll make use of both cloud and local LLMs. Lastly, we'll show how we can create and make use of an MCP server using the Kotlin MCP SDK.

[00:00:41]So, what is an LLM? Um, so as said here, type program trained on vast amounts of data to understand, summarize, generate, and predict new content.

[00:00:50]And importantly in the cons- in the context of a genetic AI and this talk, they can leverage external tools to extend the reasoning capabilities and provide provide grounded real-world utility.

[00:01:01]Another definition this time from Gemini appropriately. Um, talking again about being able to understand, generate, summarize text by analyzing massive data sets and also makes reference to the transformer architecture, uh, key component of modern LLMs.

[00:01:16]So, what is an AI agent? Uh, this is from Svetlana's excellent video. Um, describes an AI agent as a computer program that uses an LLM as its brain to solve a given task. Uses tools in a loop based on feedback from the environment.

[00:01:31]Another definition from the Kook docs themselves, an AI entity that can interact with tools, handle complex workflows, and communicate with users.

[00:01:40]And Kook then is the official Kotlin framework for creating AI agents and it's fully multi- multiplatform ready running on Android, iOS, JVM, and the web.

[00:01:49]Uh, hopefully if you haven't already and definitely recommend watching the recording of uh, Vadim's excellent talk on Kook yesterday which um, Vadim is the tech lead for KUK.

[00:02:01]So, KUK supports a range of functionality, including, for example, two lamp speed support, which we'll be demonstrating today.

[00:02:08]We'll also be creating custom strategies and demonstrating KUK's KMP support.

[00:02:12]Other functionality supported by KUK includes context management, persistence, and observability and telemetry, and we'll be showing example of the last item as well today.

[00:02:22]So, KUK supports a range of different LLM providers, including the key ones, for example, like Google, Open AI, and Anthropic.

[00:02:30]And also provides access to local models through Llama.

[00:02:34]And as of the 1.0 release yesterday, KUK also supports Light ORT Light ORT LLM integration, which allows them to support running local models on Android and iOS. The initial support in the release yesterday was for Android, but I believe iOS will be coming shortly after.

[00:02:53]So, this is how simple it is to start using KUK.

[00:02:56]We define an AI agent with a prompt executor, like Google, specify the model, and set a system prompt, which defines the agent's persona. In this case, a climate emissions specialist.

[00:03:09]To make the agent more useful, we can add tools. In this case, for example, we're adding the get emissions tool, which is going to provide it emissions data to the to the agent.

[00:03:19]Lastly, we can create custom strategies.

[00:03:21]This allows us to control exactly how our agent loops through tool calls and handles multi-step reasoning, for example.

[00:03:29]So, in in the talk today, we're going to be developing UI, and we're going to be using Compose Multiplatform. This is an open-source declarative framework from JetBrains for sharing UI across multiple platforms.

[00:03:40]Next up, MCP. MCP, which is probably where at this stage, is an open standard that allows LLMs to connect to and use external data sources and tools. And we'll We'll an of MCP later in the talk as well.

[00:03:53]And we'll be using the Kotlin MCP SDK for that.

[00:03:58]We'll also need to get data from a remote endpoint, and for that we'll be using Ktor. Uh just a quick show of hands at this stage, how many people here have used Ktor?

[00:04:06]Okay, quite a few, great.

[00:04:08]Um okay, for the live coding session today, uh we're going to start with basically the shell of a Compose Multiplatform application.

[00:04:14]More or less as if generated using the KMP wizard in IntelliJ.

[00:04:19]We'll start by creating a basic singleton Kook agent that connects to a cloud LLM.

[00:04:26]This has some limitations.

[00:04:28]We'll then start building out some local tools to sort of ground our agent with real data. And And our local tool in this case is actually then going to use a Ktor client to get the data from the remote endpoint.

[00:04:38]We also have a Ktor server in the project as a sort of a backup in case we have any issues connecting to the internet or our endpoint's not available for any reason, so hopefully we won't need that.

[00:04:48]And lastly, uh Oh yeah, sorry. The The next stage then is Okay, we we have a singleton agent, but we now want to support conversational UI interaction with the user.

[00:04:58]And for that, we'll show how we can demonstrate a custom Kook strategy and then build out the associated Compose UI code as well.

[00:05:04]And lastly, uh we'll show how we can create an MCP server using the Kotlin MCP SDK.

[00:05:09]We'll give an example of its use in Compose Desktop and also show how it can be used in Kook as well.

[00:05:14]Uh the talk today is the live coding session is kind of base loose base loosely on the Climate Trace sample that is in GitHub that has a more comprehensive UI and other data sources, but uh definitely recommend people taking a look at that if it's of interest.

[00:05:28]And shout out at this point to the excellent Climate Trace endpoint that we'll be making use of today.

[00:05:33]Okay, live coding.

[00:05:39]Okay, so uh what we're we're going to start building out uh our initial AI agent.

[00:05:45]And I'm going to be using a templates for this.

[00:05:48]So initial uh check is the font okay for folks? Particularly folks at the back.

[00:05:53]Um okay, so this is basically where we're going to construct and configure like I said our climate focused AI agent.

[00:06:03]So this is a basic agent more or less as we showed in the slides earlier. We have a prompt executor, an LLM model, and a system prompt.

[00:06:09]One key difference here is that instead of defining the prompt executor and the LLM model directly in the in the agent like this, we're actually using KMP's expect actual. And just another show of hands, how many people here have used KMP?

[00:06:21]Okay, quite a few. So you're familiar with expect actual. If you're not, this provides mechanism in KMP. Typically, you want most of your codes to exist in uh shared common code. And just to show an example here on the left, we typically have a common main in our KMP projects where as I said most most of our code goes. And this is common code that runs across all our platforms.

[00:06:38]And then we can we can create an expect where we can then define the actual implementation then in source set specific to each platform.

[00:06:46]Okay, so if we go back again, if we click on here, we can see for example that we have we're using the Gemini 2.5 flash in this example and using setting up a prompt executor with our API key for example.

[00:06:58]And we do the same in the JVM for now, but we'll show an example later where we actually customize the model that we're using on particular platforms.

[00:07:08]If we go back to our agent again. Okay, so let's do an initial sanity test of this agent.

[00:07:19]Okay, so this is a simple main function that's running in our on the JVM.

[00:07:26]This is basically going to create the agent and run this particular query, this prompt. Show emissions for Germany in 2025.

[00:07:33]So let's run this.

[00:07:40]Okay, hopefully we are uh hopefully the internet will behave for us today.

[00:07:46]Okay, so great. So, we got a very comprehensive answer here.

[00:07:49]It sort of uh provides a lot of background information. And this is information that's sort of based on what the the model has been trained on.

[00:07:55]It's sort of it can conjecture. It can kind of pick it background information.

[00:07:59]But, it's missing access to real data.

[00:08:01]So, first kind of quiz, I guess. Uh given what we've talked about so far, any suggestions about how we would ground this agent with real data?

[00:08:09]Our tools are hopefully I think I did.

[00:08:11]Haha.

[00:08:12]Okay, yeah, exactly. We can add a tool.

[00:08:14]Um so, before we can add a tool though, we need to give get a provide a way to get access to the remote endpoint. And for that, we're going to be using a Ktor client.

[00:08:26]So, this is going to be a basic Ktor client. Uh connecting This is the endpoint that we're connecting to. Uh we also have a backup. As I said, if we need to run our Ktor server locally, we have a backup where we can connect localhost to that.

[00:08:42]And with any Ktor client, we need to define a HTTP client object. And this is the object we're going to be using for making a request to Ktor.

[00:08:47]And I just a small note here about the fact that uh as I said before, we have sort of our Gradle dependencies are pre-prepared for the project.

[00:08:56]And how we actually interact with the with Ktor then depends on the platform.

[00:08:59]And based on the Gradle dependency that we've set up, it will use the appropriate engine for that.

[00:09:05]Lastly, we define our API itself at it.

[00:09:08]So, simple simple method we're using our client object. We would call in get method.

[00:09:12]This is the endpoint we're hitting.

[00:09:13]We're providing a country list and the year to the as parameters.

[00:09:18]Making a request and we're returning the data back. So, okay. So, we have an API client now. So, now we can start building out our Cook uh tool.

[00:09:30]So, tools um as I said here, are the bridge between the LLM and the real world. And we're going to find our own for classes and classes here with various annotations, and basically Kook convert these into tool called schemas that is provided to the model. So, important point here, and we'll see this in more detail when we talk about the custom strategies and so forth, is that the LM itself does not invoke these tools. Basically, an agent like Kook will provide the necessary schema to the LM, and the LM will at some point decide whether it needs to call these tools. It will tell the agent about the invocation, what parameters needs to pass, and so forth, uh based on the information that an agent like Kook like in this scenario provides, uh but the agent itself will actually make the call.

[00:10:12]And then we have the tool itself.

[00:10:15]Okay, so uh again, some more annotations, uh just some more metadata that Kook needs to provide the LM. And this is sort of the main method we implement there, and the rest of the tool call as I said here, life cycle, schema generation, argument parsing, result routing, is handled by Kook.

[00:10:29]So, we're using our climate API client, we're calling get country emissions, and passing in the country list and the year.

[00:10:35]And as typically the case here, we're returning the result just as plain text back to the LM, which the LM typically knows how to to uh parse and respond to.

[00:10:43]Okay, so now we can go back to our provider.

[00:10:52]We can add a parameter with our tool registry uh information.

[00:10:57]So, in this particular example here, I'm instantiating these objects like climate API client and get emissions tool directly.

[00:11:03]In a more robust application, for example, in the the main climate uh sample, we're using coin, using dependency injection, for example, to inject those.

[00:11:12]Okay, now that we have that, uh we can hopefully run our agent again.

[00:11:20]And we see, okay, fine, great, we're getting Germany's emissions in 2025 are this value here. And based So, this is our agent running, interpreting what we're asking, determining that it needs to call the tool, and making this call.

[00:11:29]So, one other this is uh sort of a trick question to some extent. Uh is there anything in the response there that looks incorrect?

[00:11:41]Okay, so I think I heard I'm not too sure, but one important thing about LLMs is they don't really have a great grounding in terms of date and time, the current date and time. They're typically based on the sort of when they were trained. So, you see Germany's emissions in the 25 are this. So, or you sometimes you see are projected to be. So, typically with an agent like this in in the main sort of sample, this is based on I have a specific tool for getting the date and time.

[00:12:01]But something like this we can say uh the year is currently if I can type that right, 2020 26, I think.

[00:12:09]Um use data up to and including this is hopefully the most typing I have to do in the in the talk. 2025.

[00:12:17]And if you run this again, we should hopefully see the the very important uh past tense. Okay, Germany's emissions 25 were this. So, it's got it's got a grounding in terms of what date it is.

[00:12:27]One other change here, we're we're seeing this over very unreadable number. We can say use units of millions of of tons of CO2 equivalent.

[00:12:42]If we run this again, Okay, we we see something that's a little bit readable.

[00:12:50]Okay, before we move on to the compose code edge, we just sort of take an assessment here of what we've done. Uh we have the basic cook agent here. We have a prompt executor model defined on a per platform basis. Uh We have our system prompts which we can evolve somewhat. As I said, this sort of system prompt shapes the the system's persona and we can set up constraints there.

[00:13:07]And all importantly when it comes to sort of agent development, we define a number of tools here. We've defined a tool one tool so far here that allows us to ground our agent with real data. And we've done a sort of a basic console application to uh demonstrate that this is working correctly.

[00:13:21]Actually, one other thing I did mean to add that I do need to add here is an event handler. So, Cook supports the concept very similar you might have seen Buddy mention this making this comparison before if you used um if you used Ktor before, you sort of the idea of installing features and plugins for example, and Cook takes a very similar approach. And we initially installed this event handler plugin here or feature, and we're showing you some other feature later on as an example. So, we installed event handler, and we get we get some indication then that a a tool is being invoked.

[00:13:52]So, if we run this now again, we get our answer, but before we get the answer, we'll also see this log out here. So, we can see here that the LLM decided that it needed to invoke based on uh what we provided in the prompt, it needed to call get emissions tool. These were the arguments that it needed to pass. And as I said here, and we'll see that in more detail in the uh when we talk about strategies, uh the agent then will make will invoke that tool for it.

[00:14:14]Okay, let's talk about Compose a little bit. Uh So, this as I said is a shell of a Compose multiplatform application.

[00:14:21]Um we've uh this is sort of some default Compose code here, and this is the same Compose code that's going to be running on iOS, Android, and JVM in our project here.

[00:14:31]The We we can Compose multiplatform also supports the web, and then as I said, the broader example includes web, but for day today we're focusing on iOS, Android, and JVM.

[00:14:42]Okay, I'm going to add some initial UI here uh just to sort of I'm going to build we'll explain what it's doing, and we'll uh talk a little bit about some of the next steps we're making. So, uh another show of hands, how many people here have used Compose?

[00:14:57]Okay, quite a few as expected. Again, if you haven't, Compose is sort of a declarative UI framework, and what that generally means is that it's very state-driven. You define certain states, and you declare what your UI should look like in that state. Any changes in the state would cause your UI to be re-rendered.

[00:15:10]So, in this basic example, and as I said, we'll be evolving this, and we have some sort of temporary code in here, but right now we have sort of hard-coded uh inputs, get emissions for Germany, we have something indicating that the agent is running, and we have the result. Um we're also creating the agent directly here as well.

[00:15:30]Not something we would do in a sort of normal application, but just for ease of illustration, we're going to uh construct the agent directly.

[00:15:37]Again, we also sort of as a temporary measure, we have this launch effect here, which is a piece of code that will run when the UI is shown.

[00:15:42]And in this example, with sort of our initial hard-coded prompt, we're going to run the agent. This will result in this result state being updated, which in turn will result in this text element being rendered with the results. And some indication here are using a progress indicator to indicate that the um the agent is running.

[00:15:59]So, let's hopefully run this.

[00:16:05]Okay.

[00:16:11]Okay, and we see the first limitation of a single relation like this. We've asked it for emissions for Germany. We haven't specified the year.

[00:16:17]It needs to know what the year that we're asking for, and we've no way We've no means to tell it. So, but for now, I'm going to go back to the UI. We're going to build out the UI a little bit more before we move over to sort of supporting that conversational interaction. And I'll put in the specific date here.

[00:16:34]We run again.

[00:16:38]Okay, at least we see the data that we got back before.

[00:16:41]Okay, before before moving over to the sort of that conversational interaction, let's just build out the UI a little a little bit more. We want to show sort of a chat interface with sort of different chat messages.

[00:16:56]So, we're going to have We're going to represent that We're going to have data class like this representing a chat message in our UI.

[00:17:01]The message is either going to come from the user as sort of the prompt or or a reply to the LLM for some some clarifying question it might have.

[00:17:07]We have something coming from the assistant, which is from the LLM typically, and some tool invocation, for example.

[00:17:20]>> Okay, we have a list of messages and that's going to represent the list of messages which we're going to show in our lazy column.

[00:17:25]Um but to to populate those to to sort of provide the data that we need for that, we need to add some sort of callback to our agent from our agent.

[00:17:33]So, we're going to add a couple of parameters here, some indication that a tool is being invoked.

[00:17:41]And sort of placeholder for now, the result of this is a single run agent and the result of the agent will be returned and that's what we're rendering in our UI right now, but ultimately when we create a conversational interaction, we we want to some callback so that we get intermediate callbacks or intermediate sort of responses from the agent at different points. So, that's what our on assistant message will be used for. But for the time being, we'll at least um get something for the tool call.

[00:18:08]Okay, we can now go back to our UI.

[00:18:11]Okay, so we've provided out of this, we need to add something represented.

[00:18:15]I don't know. Siri has decided to jump in for some reason. I'm not sure why.

[00:18:20]Must have said some keyword.

[00:18:22]Um Okay, so we need to provide then the uh parameters.

[00:18:28]So, nothing too complicated here. We have our list of messages. When we get a tool call event, we'll add a tool message to the UI.

[00:18:35]This again is a on assistant message as a placeholder right now. We'll come back to that in a little while.

[00:18:39]And for now at least, we'll update the sort of this somewhat temporary code right now in the single run agent scenario to um to basically run the agent as well, but provide the sort of the user and the system message into the UI. And as I said, this is sort of somewhat temporary. Like when we get to the point where the user is providing their own input or when we're getting some conversation direction going, this will be no longer be shown.

[00:19:03]So, instead of showing a single text element now, we need to show something with the lazy column.

[00:19:08]Okay.

[00:19:09]And I think we're ready to run this again.

[00:19:14]Oh, yes. Okay, we need to provide something here for our console application.

[00:19:26]Okay, we're looking starting to look a little bit better. We're we're seeing some of the key life cycle. We're seeing the prompt being provided by the user.

[00:19:31]All of these hardcoded for now. Seeing tooling invocation and we're seeing the final result.

[00:19:35]Uh and so before we move on to the conversational interaction, I just want to build out our UI a little bit more.

[00:19:46]I've created specific composable for showing each message that allows us to sort of put some kind of label as to what type of message it message it is, what sort of color code it somewhat. And importantly, we're starting to provide we're we're starting using this markdown here. And the markdown basically, it's nice library from Mike Penz.

[00:20:01]And the reason we're using it it is that LLMs, as mentioned here, naturally produce markdown. So, this markdown composable will render the LLM response with full formatting as mentioned here, bold, tables, bullet lists, etc. We'll see an example of that later.

[00:20:19]Instead of using text here now, we will use the message view.

[00:20:27]We run this again. Okay, we get something that looks a little bit better.

[00:20:31]Okay, so now if we switch to sort of what we need to do is support sort of conversational interaction with the user.

[00:20:37]What I alluded to before is the fact that we need to create a custom strategy for that. So, if I go back to the provider, we can add a a parameter here now to our agent, which is If I get this right, yes.

[00:20:54]Okay, so we can create a strategy that the agent would use. Um but before showing the actual strategy that we're going to use, I'm going to actually pull in here what effectively represents the default strategy that the agent would use. And what we're going to create is sort of a small enough evolution of that, but it's informative just to see how that works.

[00:21:11]So, a lot of code here, um, but this kind of this loop, this this, uh, dance that happens here between the agent and the LM is kind of important to understand. So, typically what you have happens here is that for a given input, we send the input to the LM. Uh, the LM will will decide if it needs if there's tool call information or we Sorry, we check the agent rather we check the LM response to see if there's tool call information required by the LM.

[00:21:36]If there is, we extract that information, and then we execute the tools. So, at least the agent executes the tools. As I said, the important point here is that the LM itself doesn't execute the tools, but based on the information that the agent provides the LM, the LM will decide will decide whether to call tools or not, will provide that information to the agent, and then the agent will invoke it.

[00:21:53]We take the information we get back from our tools, then we pass it back to the LM again. The LM again may decide at that point to call more tools, uh, so we check that again.

[00:22:01]And we keep looping until we get basically a, uh, text response back.

[00:22:05]Okay, this is so-called assistant message.

[00:22:08]So, let's run this again.

[00:22:10]And we should see exactly what we saw before. Okay, so now we're using what is effectively the, uh, the default strategy that the LM the the that Cook has under the hood. Just a small point in terms of the syntax here and what we're using, we're using functional strategies here.

[00:22:24]If anybody saw Vadim's talk, he was using the sort of a grass the approach of course using a grass strategy. Um, depending on what you want to do, depending on how sophisticated your strategy needs to be, there there seems to be some suggestion No, seems to be Sorry, there's a suggestion I guess that the graph strategy is a little bit more flexible, but again, it depends. I prefer the functional strategy. I think the the code is a little bit more readable and easier to understand.

[00:22:46]Okay, so now let's move over to our, uh, to a strategy that supports, um, conversational interaction.

[00:22:58]Okay, so, uh, a lot of code here again, but fundamentally it's the same as what we saw before which with an outer loop now.

[00:23:03]So, we have our inner loop before we sort of handle the genetic tool use.

[00:23:06]And now we have an outer loop basically that will run as long as the user keeps sending messages. So, we have this outer loop here.

[00:23:13]And this basically sample here will keep looping until the user provides an empty string for example.

[00:23:18]Uh but the inner loop is the same as before with the the sort of management of the genetic tool use.

[00:23:24]A key thing here now is that the response each time it runs it when when it gets a a system message that it wants to provide to the UI it now starts to use this on a system message callback and that's a suspend function. And a key part of this interaction here is that that will suspend until the user provides an answer or a subsequent prompt and we'll see that in more detail shortly.

[00:23:42]Okay, now we have that our custom strategy there, we need to provide the on a system message callback that we provided. We kind of made reference to it earlier.

[00:23:51]And now if you go back to our UI uh we can start to sort of build out that that a little bit more.

[00:24:00]Uh So, we're going to use a Kotlin channel basically. So, user replies a Kotlin channel. Basically, this is what's going to provide a mechanism to bridge the UI to the suspended agent strategy.

[00:24:11]We'll see that in more detail here when as we add in the code for the on a system where able to add in this code with the on a system message callback now.

[00:24:22]So, key thing here uh so, we get the response from the LLM from from the agent. We call user reply user reply.receive here which is basically going to block until user reply.send is invoked. And as we'll see in a moment when the user provides a response or subsequent prompt, we will call user reply.send.

[00:24:39]That will unblock this and the strategy loop will continue with the user's reply.

[00:24:45]Okay.

[00:24:46]We can happily get rid of this code here now.

[00:24:50]And we can add in an edit field then.

[00:24:57]Okay, so we now have an a text file, standard compose text file where the user can now enter their as I said prompts and replies.

[00:25:03]And that's going to invoke on send message.

[00:25:10]So, small bit of a back here. Uh we're again we're adding to the list of messages that again with the user prompt.

[00:25:16]And basically what we do here depends on whether it's the first time or not. If it's the first time then we want to kick off the agent loop. We saw that strategy where the where we loop continuously our loop until at least the user provides an empty string.

[00:25:26]And the key thing here is then in subsequent messages then we just send into the channel and the suspended agent then receives that and continues the loop.

[00:25:35]Okay, so I think we are almost in a position to to run this, hopefully.

[00:25:41]Okay, so now we we have a prompt here.

[00:25:42]Let's remove Let's remove the year.

[00:25:46]Screams true. Okay, and then the we see okay, it needs the it needs the year so the user can we can reply with the year.

[00:25:51]We have sort of context and history here and we can add things like um and France.

[00:25:58]I didn't know it was the context of what we mean. We We mean emissions for France and we mean in 2025 because that's what we've already been talking about here.

[00:26:04]And let's see show in a table.

[00:26:11]Okay, we see the markdown support for example here.

[00:26:15]But uh let's see we What if we want to compare per capita numbers?

[00:26:22]Uh I misspelled it but spelling doesn't matter. Well, then it's Um okay, it can't because it doesn't have access to population data.

[00:26:29]So, another small quiz here. Based on what we've talked about so far, how would we address this? How would we provide population data to our agent?

[00:26:36]Tool calls, exactly. So, let's go back to our agent.

[00:26:43]And as I say something prepared earlier, we have a get population tool and a get population client. So, the population client is again another Ktor client where we're connected to the endpoints to get the data.

[00:26:56]And then the tool is again, as we saw before, just using Ktor basically to wrap that around, providing information to the LLM for example in terms of what it expects in terms of country code and so forth.

[00:27:07]Now, if I run this again, hopefully the LLM now now knows about population data. So, if we uh compare per capita numbers for Germany and France.

[00:27:24]I will pick on other countries shortly, so don't worry.

[00:27:26]2025.

[00:27:30]Okay, so we we get the emission tool as before, but we also see the get population tool being invoked. And let's see again, show in a table with population data. So, we see that.

[00:27:46]Okay, and then we get to see the data here. So, we see the more meaningful per capita numbers, and we see it nicely laid out in the table and so forth.

[00:27:52]So, uh okay. So, I'm going to move shortly over to the MCP server. There's a few more things I want to show here.

[00:27:57]Um so, I talked about installing features for Ktor.

[00:28:01]So, another feature that we can install is um OpenTelemetry. Uh it's a very nice capability, and I'm going to be using sort of the line fuse integration here.

[00:28:12]And um also want to use this as a as a means to again show KMP mechanism in place here. So, install platform agent features is an expect function.

[00:28:22]Uh right now in this in this example here in the iOS main function, we have uh an empty implementation.

[00:28:30]But, if we look in JVM we see that we're using a line fuse integration.

[00:28:36]And uh just a small point, as of prior to yesterday and using the version I'm using here, that integration was only available on JVM. Um Thanks to Sergey Dubov from the Ktor team. I was kind of giving him a hard time earlier for ruining my demo. Um he actually implemented support for OpenTelemetry across all the platforms.

[00:28:54]Um so, if you're using the 1.0 release, uh you can use very very It's great to see you can see the OpenTelemetry integration running for all the platforms as well.

[00:29:02]Okay, so but in this example, this sort of hopefully demonstrating the sort of KMP's expect actual. We see this is, you know, it's useful capability to be able to provide per platform uh features for our Ktor agent.

[00:29:13]Okay, so a little bit more participant uh uh audience participation because I'm going to be looking this shortly in uh LangFuse. We're still calling it the data anymore, so I'm going to stop picking on Germany for a while. Anyone to shout out any other countries that they're from who could shoot to use?

[00:29:28]Sorry?

[00:29:29]Italy, okay. Italy. And anywhere else?

[00:29:34]Zimbabwe.

[00:29:35]Oh, cool. Okay.

[00:29:36]And then Hopefully, I can spell this right. I'm probably not.

[00:29:41]Apologies if not.

[00:29:43]Is that okay?

[00:29:45]Okay.

[00:29:50]Okay, great. So, now I can switch over to I can hopefully switch over to um do a refresh here.

[00:29:56]And we see uh let's see if this is data.

[00:30:06]Okay, no, it's not being updated yet.

[00:30:09]Maybe we're hitting uh sessions, tracing.

[00:30:13]Ah, here we go. Forward. Okay, yes.

[00:30:15]That's a relief. Um okay, now we can So, based on that I saw that feature, uh we've enabled that uh LangFuse integration. And we get this nice telemetry here. Uh we see, for example, uh the fact that we're using the 2.5 flash um that we saw configured earlier. We see the get emissions tool with Italy and Zimbabwe.

[00:30:34]And and we see the final result coming back from there, for example. So, very nice. You you get to see uh uh key emissions to so latency, token uh cost, for example, and so forth. So, these are capabilities I have.

[00:30:48]Okay, so um I want to do one other thing here, which is uh we mentioned earlier that we can define per platform uh models, for example. So, I'm going to switch over to a local model here uh using a llama. Has anyone used a llama here before?

[00:31:07]Okay, a few folks, great. So, llama, obviously, if you haven't before, uh I'll just run this here, hopefully locally.

[00:31:15]Allows you to install download and install uh local models running on, in this case here, for example, the laptop.

[00:31:21]And these local models have become a lot more powerful. Um not just for scenarios like this, but also using with coding agents, for example, you can use a llama with cloud um if you want to um and they're getting, you know, comparable. I think there's a little bit of an arms race there.

[00:31:35]They're probably always going to be a little bit behind the cloud models in terms of at least for some of the coding scenarios.

[00:31:39]But I I I pick this particular here.

[00:31:41]It's a grand 3.5, 35 billion parameters, and sort of mixture of experts approach, so only 3 billion parameters use at a particular time.

[00:31:50]And it's optimized for the MacBook as well. So, it's it's quite performant. It runs at similar takes similar amount of time to some of the cloud LLMs.

[00:31:57]So, if we run this here again, and let me pick another country. I'm going to pick Ireland.

[00:32:04]Why not?

[00:32:09]Okay, so it's a little bit slow at the start. So, basically, if I switch over, hopefully, to activity monitor, you see I see llama here. So, a key constraint, I guess, with running these models is the amount of memory they use.

[00:32:19]So, this is a relatively small, well, in in comparison to some other models, but it's using about 20 gig of memory, so which is thankfully I have enough space for here. But it takes a little while at the first first request, but then uh I'll put Belgium here, for example.

[00:32:35]It's quite fast on subsequent queries.

[00:32:38]And hopefully I can go over now to um line fuse.

[00:32:43]Refresh. Do we get our result back yet?

[00:32:45]It's a little bit slower today.

[00:32:48]Yes. Okay, there we go. So, we see the the grand model being used. We see get a message being tool being called with Ireland and then later with Belgium. We see the result coming back as well.

[00:32:58]Um can anyone see anything missing from here that we saw when we were using the Gemini?

[00:33:04]Cost. Yes. So, if we go back to here, we can see for example this is how much this cost.

[00:33:09]Obviously with the local models, uh they're they're both free and private, which is which is nice.

[00:33:14]Um okay, and you know, it it's a fascinating area in and of itself. I mean, it's it's we weekly announcements these days in terms of new versions, new new evolutions of these models. Um being more performant and being more capable.

[00:33:28]Okay, so uh where are we? Okay, so let's see we're going to switch tack a little bit. I'm going to talk about MCP.

[00:33:40]Get my live templates hopefully in the right way.

[00:33:43]Okay, so um MCP as mentioned is an open standard from Anthropic uh that lets any LLM host discover and call tools over standard protocol. Uh you sometimes see it described as USB-C for AI tools.

[00:33:57]In this case here, we're writing an MCP server that it it's going to expose the Climate Trace API as an MCP server. Um we're going to be running this as a different protocol. We're going to be running over what's called standard IO.

[00:34:05]It's going to be running as a local tool.

[00:34:07]Uh local sorry, MCP server and then we can also deploy it as a remote server.

[00:34:11]In the in the broader sample that I have, I have this I have an MCP server that's deployed remotely into cloud run as well.

[00:34:17]So, this is going to use Climate API client. Uh this is the same uh object or class that we use from our cool tools and it's running in shared can't be code as well.

[00:34:29]Uh let's see you can figure server. Oh, no, sorry. Okay.

[00:34:33]No, I'm in the wrong place.

[00:34:37]We have our get emissions. Okay, so we're going to use that. So, first of all, we create the server object, which is sort of a a object we can use to sort of add tools to our MCP server. Uh just to mention a small thing, MCP supports tools, resources, and prompts in general, but I think most people most usage you see is with tools. But, so in this case, for example, we're using it with tools.

[00:34:55]Uh we're now at the get emissions tool.

[00:34:57]Um there's quite a bit of boilerplate here. This is very similar we're effectively doing the same thing as we did with our local tool earlier with the code.

[00:35:02]Uh we just need to provide a little bit more boilerplate for it to define the schema. And again, this is the information that the SP host in this case will provide or will pass to the LM.

[00:35:14]And then we have the body itself. And again, a little bit more boilerplate uh parsing out the arguments.

[00:35:19]This is the key call here. We're using our client API, get country missions.

[00:35:22]Again, the same call we would have made in our tool earlier.

[00:35:25]Um and then call tool result and we'll wrap the response as a list of text content blocks and the MCP forward MCP host would forward these back to the LM as tool call results. Again, as we saw for example in our strategy earlier with code.

[00:35:37]And importantly then, we need to add a main method here.

[00:35:40]Which will basically set up our our standard IO transport.

[00:35:46]Okay, I'm going to use shadow jar. Uh hopefully folks have seen that. It's sort of gives us the ability to we're going to create a sort of executable jar file that has all the dependencies included.

[00:35:55]Okay, and before we switch over now to um cloud.

[00:36:00]I'm just going to do a sanity test on the MCP server using very nice tool called the MCP inspector.

[00:36:09]Okay, as you can see I run this before, so it sort of has set up already with the sort of parameters here. We have Java jar, and this is the jar file that we just built a moment ago. Um Okay, hit connect and happily see that green light come on, which is good.

[00:36:23]Uh we can list our tools. We have the get emissions one, which we saw a moment ago.

[00:36:28]Country code list, let's try this out.

[00:36:30]We have to provide the code specifically here.

[00:36:33]2025 and run the tool.

[00:36:37]And we see that same data we've seen a number of times coming back. Okay, so that's a useful way to do a sanity test at least the MCB server. What I want to do want to do is demonstrate this in Cloud Desktop initially at least.

[00:36:51]So before we run this, just if we look in the Cloud settings and under developer settings, you see obviously that I pre-configured Cloud already with the same information.

[00:37:01]There's a sort of configuration associated with the Cloud configuration file, sorry. So to the Cloud and we we can add our MCB servers in there. So this is the same JAR file that we just built added into Cloud.

[00:37:10]So now let's go back here and say something like, okay, get per capita emissions for, let's see, Nor- Norway. I'll pick on Norway cuz there's my employer is there.

[00:37:24]Sweden, let's do and Finland, say.

[00:37:28]If I can spell again, in 2025.

[00:37:31]Okay, so it's going to go off and get that information. Hopefully what you can start to see here is loading tools and get emissions. So as it's sort of loading loading that information, we can see that it's referring to the Climate Trace integration. It's using our MCB server.

[00:37:45]You can see get emissions called. It's figured out the obviously the three country codes that need to be provided, the year, and the results.

[00:37:51]And we get to see some we get to see the results and with some of the nice sort of capabilities in the sort of recent versions of Cloud Desktop, you get some nice infographics as well. And for example, in the in the broader client again referred to, we can also see things like for example, the um what do you call assets? The The What's what's contributing to these emissions in different countries and sort of navigate down to those. So very useful capability to have.

[00:38:16]Okay, I'm doing okay for time, I think.

[00:38:19]Okay, so let's go back to uh so Actually, by the way, I forgot to switch it out, not that it really mattered, but this is again using that local model.

[00:38:29]It's using the using the Gwen model to to make these queries.

[00:38:34]Okay. So, let's go back here again.

[00:38:38]Minimize this.

[00:38:40]And this is the risky part of the talk.

[00:38:44]If I can find my nice little image.

[00:38:46]One more thing.

[00:38:48]Um I mentioned earlier that uh Kook 1.0 was released yesterday, and one of the very nice things about that is that Light ORT support, allowing us to run local models on our uh Android and iOS devices well, initially Android and soon to be iOS.

[00:39:04]Um so, I was I didn't want to switch to uh new version of Kook the day before the talk. Um so, I created a I made some changes in a branch here.

[00:39:14]Uh let's see. Let's get stash for now.

[00:39:18]Light ORT to support that. Uh switch So, this this uh let's make sure that we have uh Okay, our our Gradle um while Gradle is hopefully syncing uh there, uh we'll switch over to Okay.

[00:39:39]You want me 2 seconds. Need to find the right place. Okay, platform Android's great.

[00:39:43]Okay. So, now with that with the 1.0 release, uh we now have uh the ability to use Light ORT configuration. So, now in my actual for Android, I have using the Gemma 4 2 billion parameter model. I'm using the executor executor here. The executor's configured to this path on my device. If I if I click on or I think if I hold this over this, you can see the the Hugging Face URL there. So, you can go to Hugging Face, for example, download a particular model, upload it to your device, uh configure where where the location on the device where that is running.

[00:40:17]And uh so it that it runs basically runs a local model on your device in that case.

[00:40:26]Okay, so licenses.

[00:40:33]Okay, so this is the part that is hopefully will work okay.

[00:40:37]Um hopefully I think I have on my on my notifications switched off.

[00:40:41]Um and if we switch to the compose app to the actual Android application and run this.

[00:40:57]By the way, this is as mentioned before the same compose multiplatform code that we're running on iOS, Android, and JVM.

[00:41:03]Okay, so get the emissions for Germany.

[00:41:06]So um takes a while the first time. Similar to what we saw with the llama that sort of the first request requires the model to be loaded. Um it's running on our It's somewhat more constrained than our laptop.

[00:41:17]Typically seeing this take sort of um 15 20 seconds for the initial request.

[00:41:30]Yes, okay. And we get back the response and we can then do 2025 again.

[00:41:35]Now just again a caveat here again is that So this came out in the 1.0 release yesterday. It's an initial early release. Uh there are We see the tool invocation by the way happening here as well.

[00:41:45]Uh so we we see our tool being invoked.

[00:41:48]Um we see the response coming back as well. So that's good to see. There are occasional There are some limitations in the version that they're being used that's being used right now in terms of uh some of the parsing of tool responses, handling multiple tools, and so forth.

[00:41:58]Um but it's very cool to see this uh available now. Um this is supporting Android right now.

[00:42:04]Light or T works on uh iOS as well for example, and I believe there will be a not in the not too distant future release of Cook that works on um iOS as well.

[00:42:14]Okay, it's nice to have worked.

[00:42:15]Um okay, let's go back to our 2 minutes left.

[00:42:20]Go back to our slides.

[00:42:24]Okay, so what did we cover today? Uh we created a basic single turn AI agent and we to the science test, we ran that using a console app. Uh We there was some limitation with that, of course, so we grounded our agent with real data. We created a Cook local tool and used the Ktor client to get data from a remote endpoints uh for the tool.

[00:42:41]We created initial basic compose UI application. Uh demonst- running this single turn Cook agent.

[00:42:49]We then added sort of conversation UI based on a custom Cook strategy.

[00:42:53]We expanded the tools to support populate population data.

[00:42:57]We added the telemetry feature, also demonstrating use of expect actual.

[00:43:00]We used a local model on JVM. Uh we created an MCV server and used in cloud desktop.

[00:43:06]And as a bonus, uh we demonstrated the uh new uh Cook light or T uh support on running on Android using a local model, using a Gemma uh model on Android.

[00:43:16]And just a quick recap on what we built out today.

[00:43:20]Thank you very much.