Behind the Scenes of new features released for Dynatrace Intelligence | AMA Livestream

[00:00:03][music] Heat. [music] Heat.

[00:00:23][music] >> [music] >> Okay. So hello everybody welcome to our LinkedIn live session. This is our last session on Dinetrace Assist. When we very first got started it used to be called Davis Copilot but now we moved along the whole year doing a series of this and now we are at Datrace Assist.

[00:01:10]So as this is our going to be our last session and we're going to be talking a lot about all the new features that were introduced. We have quite an entourage here that is here to talk about all the cool new features, what got announced and how you can use Dinatrace SS to the best of its capabilities and what we did behind the scenes around all those environments or all those iterations of providing some really cool services for all our customers. I'm Safia. I'm the field CTO for public sector at Dinatrace. And this was a brainchild for me where I wanted to learn how our product team goes. iterates upon design ideas and then gets it out to the customers. So, I'm super proud of the fact that we went through the iterations and now we are at our last one. So, Gabby has been my uh partner in crime to say the least and I would love if you can go ahead and introduce yourself Gabby.

[00:02:02]>> Yes, of course. So, I'm Gabby. I am the product manager responsible for Dinatrace intelligence generative and agentic AI. So we've brought Davis co-pilot from a very rudimentary point solution based on Alms to something that is so incredibly powerful that we are all super excited to show you like what is new with the latest stuff share a couple of our favorite use cases and even give a sneak peek on what's coming up in future.

[00:02:27]>> Always super excited to learn about the future and then I want to give an opportunity to Thomas to introduce himself.

[00:02:33]Hello everybody. My name is Thomas Lleger working very closely with Gabby and Wolffkang to um build the technique behind the Davis assist in a way that it's future proof and extensible and hopefully in the future bringing even more new cool stuff in.

[00:02:50]>> Okay, perfect. We love having you here Thomas. Your insights and the details that you bring to the table are incredibly appreciated and we love listening from you. So Wolf Gang, over to you now.

[00:03:01]Thanks Sophia for organizing all that and for inviting us. My name is Wolffkang and I'm working together with Thomas and Gabby on and owning uh the topic of data intelligence capability at Dino Trace. So and I will present agentic workflows uh today.

[00:03:19]>> That is super exciting Wolf Gang because I know that a lot of the people don't know what agentic workflows are and how they can use that in terms of observability. So super happy to have you in here and then let's get in. We have a lot of content to go through. So Gabby, let's dive into the presentation that we have prepared for our listeners.

[00:03:39]So I do realize that there were a lot of things that were announced over the last couple of months which is um four new things that we announced all together.

[00:03:50]Do you want to go in and start with the biggest one that we announced which was like the dinet trace assist migration that happened from GPD40 on Azure to cloud sonnet and then on Amazon bedrock.

[00:04:04]So we want to understand for people who are very new and don't know a lot about AI why does the foundation ma model matter so much for all um of din trace assist?

[00:04:18]Yeah, absolutely. So, one of the things to notice here, um, if we just head through to the next slide is by changing the foundation models, we've literally upgraded the brain that drives everything behind assist and everything behind our agentic AI. So, the difference here is really like night and day. It is a massive change in terms of smarter reasoning that is now possible.

[00:04:41]So, we've got stronger multi-step reasoning. There's much um better tool use and tool selection than what we had before. Um, and the answers are so much more complex and they're so much more reliable.

[00:04:53]So really the big difference here is that while user can still be asking the exact same question, the answers that get back are just so much better and so much sharper than what they were before.

[00:05:05]And this is all thanks to using much better um, foundation models under the hood.

[00:05:11]>> Yep. Certainly. So what I'm thinking is that this was not just a swap for the sake of swapping it, but can you like walk us through what was the internal evaluation process that the product team took? What it looked like? What were you testing? Who all was involved in the testing and what did the data show while you were doing this to ensure that this is the right call or this is the right model? How was that selection done?

[00:05:39]>> Yeah. Um if you don't mind I will show you some of the very early things uh when we started out to investigate uh these migrations. Um those people which are more into the whole area of um aentic there is this very beloved word evals where basically um develop your agent harness and run it against a established set of questions and expectations about these questions.

[00:06:14]And that was actually uh some of the first things uh we extended from um the old assist where we already had this kind of possibility to use dino trace for our own observability of our own implementation um of the uh agent harness which is powering u assist. And what we do here is we have the possibility to internally configure various agent harnesses configurations by defining uh different agents here.

[00:06:46]Here are for example in this screenshot at the very top and it's not super big.

[00:06:52]You can read cla46 MCP deep agent. So what it does, what you can read off there is we have been testing a um a certain type of agent agent harness in combination with a certain MCP server in combination uh with a dedicated model and the whole combination of those things uh make the the harness work as it works right now. So a very important part as Gabby mentioned is the model but not only the model. The model needs to get context from data and the context from data is coming from various sources. Uh an important one we already had before that release was our MCP gateway but now with the deep agents we will talk about this a little bit later.

[00:07:38]We also bring in skills and then you need to have a capable brain uh integrating all those data sources in uh in nice context uh to make it work and with the observability we built in uh our own agent harness. We are now capable of uh tracking various aspects um of agent performance. um typical aspects like does it actually select the tools we interest interested in? Does it actually select the skills we are interested in? Uh does the output meet the expectations from our subject matter experts. For example, when it comes to analyzing um previously known Davis problems or does it analyze uh questions around Kubernetes properly? uh we have our subject matter experts which have expectations to certain questions and then these questions are uh cast into appropriate evaluation data sets and then we use yet another LLM as a judge to uh evaluate whether those questions uh meet the expectations of the subject matter experts and these expectations are then uh come up in a um are measured by these kind of LLM as a judge uh scores like the completeness or correctness score of various runs of various agent combinations there. Yeah.

[00:09:03]So that's uh a very important part uh you need when you uh let's say have so many possibilities to explore model combinations uh agent types data sources tool uh selections and whatnot. And uh if you would uh just go to the next slide please. Uh what's a very important part of that obviously is that you have a proper machinery uh in place under the hood um which you can use for continuous evaluation internal evaluations there um which basically uh needs to have components like you would it should be more or less um agent harness agnostic such you can run your own agent harness which you're evaluating and then do a head-to-head comparison with some implement mentations in a cloud code or open code agent harness. You need to uh collect the actual thinking process the trajectory of the agent took to give you the uh correct answer. then you need to uh to score those trajectories having multiple configurable ways in place uh to come up with certain scores like I showed in the other slide uh LLM as a charge score or more deterministic scores like uh let's say uh correctness score per spent effort per spent mega tokens for example there's also a very interesting trade-off uh you can also see these kinds of measurements coming out from entropic where they say sometimes it's it's actually cheaper to use a a better model where an individual token is more expensive but it doesn't take so long to figure the correct answer out. So that's very interesting metrics to take into consideration and then obviously as we are a observability company we will leverage our own uh uh grad data lakehouse to store all those uh results of this trajectory evaluations uh build nice dashboards and uh anomaly detection on top of that as soon as something goes wrong here uh our builds will stop and we will not uh ship uh the next release.

[00:11:09]Yeah. So in a on a big ballpark that's uh was the machinery which we finally used to decide with which configuration to go and uh based on those measurements a sweet spot in let's say um brain power versus token spent per correct outcome was that we finally went with son 4.6.

[00:11:35]Yeah >> that is incredible Thomas. Thank you for putting all that into perspective because this is not like your regular AB testing like we were talking about because um AIS are so non-deterministic.

[00:11:48]It's hard to predict their responses and every time you get a new response. So you have to make sure that you have the right use case optimization and model optimization to understand which one is the right one for our use case and what we will be using them for. So thank you so much for providing all that context.

[00:12:07]Um I think now moving on if I am an S sur or a platform engineer that has been using Dinatrace assist even today what would be the most noticeable difference that I will feel I know that the brain has changed I know that uh we do have a new way of thinking or a new mod module in there or a new model but what would I see as an enduser in order when I go in and try to chat with Dinatrace Yeah. So, Safia, um I had such a hard time actually picking just one use case to show off today. Um but what I actually wanted to show is specifically um one of the examples that we used to get a lot of let's say very mediocre feedback on. So whenever customers used to ask us about you know something like problem overview or root cause analysis the the responses with our previous model and our entire previous setup was a little bit lackluster. So the example I prepared for us here says review all active problems in my environment. Find the most critical one and perform a root cause analysis including affected services impacted uses and correlated log patterns. And I wanted to pick this one because it just shows how the the experience has genuinely improved so significantly from before due to a combination of all of the changes that we released. So I picked this example because it is specifically one where previously this is one of our go-to use cases for customers. And like I said the the the the responses they didn't used to provide the same rich detail that they do now. So if we have a quick look at what happens here is we can see the exact thinking process of our AI. It's reading different files which we will get back to just now. It's calling all of these different tools under the hood and then what we get here is a significantly richer and deeper analysis of what has been going on here. So it gives you an overview of all of the problems in the environment. It tells you you've got 35 active ones spanning all of these different categories. And then it dives much deeper into this one specific one.

[00:14:20]So we now see, sorry that one just jumped on my screen. We have this active um availability error problem. Um and then it gives you a full root cause analysis. It gives you the full overview of the affected services and entities exactly like I asked for. And then it also gives this overview of the actual impacted users and then the correlated log patterns. And I absolutely love this because this is actually one of the new um analyzers that we released over the course of the last few months. Um and it really it's just completely changed the way that people are diving way deeper into um all of the data that they have available. So this is super rich and it helps so much with providing even deeper insights especially to uh this kind of a root cause analysis.

[00:15:07]And what is also so much better now is the recommended actions. So you can see here instead of having some generic actions that kind of tell you, you know, go check out this one particular page here or there, we now have these um these these actions where it tells me do the following immediately, do the next in um the short term and then you can also follow up with these two suggestions over here. So the entire experience is much richer and it is simply so much um more relevant to all of our customers. So I hope that gives you a quick overview of some of the the big changes that we are seeing thanks to having the new more powerful foundation model.

[00:15:51]Certainly and I have been one of those people who have been frustrated with the responses because a lot of the people what they do is when they come to Dinatrace ess they also have either clawed or something else open and they are using the latest and the greatest models and we could see that sometimes the responses were not very accurate or detailed. But then coming to this what you exactly showed Gabby which is having all of that information it having all the multiple tools in order to go in look at files u call all the tools as and how it requires and then being able to provide that comprehensive response is an incredible feature for our customers to be able to experience. Um so great job on switching up the model.

[00:16:35]I guess it was it it chose.

[00:16:39]>> Yeah.

[00:16:41]>> Okay. So, um we'll keep moving on and my next question is I'm sure when you are migrating the models um in a product like Dinatrace which is incredibly difficult and there is a lot of complexity, what are the things that broke like what are the things that surprised you or what are the things that took longer than expected? because I do realize that we have been talking for this migration since our very first um uh when we did our very first LinkedIn live and now at the last one it's finally made it into the product.

[00:17:14]So just understanding what that whole structure was like and what was the biggest thing that you had to solve in order to get this whole thing working.

[00:17:24]>> I mean what was the the the big question? What was the the biggest thing we had to solve? I mean it it was the the sum of things we had to solve. I would argue um because you know Dino trace ships is ships its SAS offering in I don't know exactly how many regions all over the world and uh we have to deploy and uh leverage these foundational models in a secure data privacy data location aware manner uh to uh can give um SAS level SLAs to our uh customers uh which makes sure that data is processed where they expect it to be processed. It's stored where they expect it to be stored or not stored actually and that uh that the um authentication and integration with our own IM services is uh worked out in a way um that will scale in the speed the offerings uh of for example of AWS bedrocks are scaling.

[00:18:30]So early solutions uh which were referenced back then as reference architectures from AWS itself uh would have turned out in a way that whenever they bring up a new capability within uh um Amazon Bedrock, it would require them would have required us um non non significant code changes in the integration. for example depicted in this image in those integration lambdas uh which you need to set up in uh behind an API gateway to make sure you have proper uh streaming and endpoint reachability and whatnot and um to have this set up in a scalable way we worked with our colleagues and uh uh partners from AWS to come up with a scalable AI infrastructure deployment and if you're interested in there is a blog post out there from AWS which you can look up how we actually uh developed uh a new reference architecture to integrate uh infra uh AWS um AI infrastructure into your SAS offering with a dedicated back call to our to your own um to your own IM IM offerings and in a in a way uh that your engineering of your own agent harness doesn't require you to rework the uh infrastructure all the time. And with such an infrastructure in place, we are now uh in the position uh with a new when a new model is coming out. For example, recently Son 5 got released uh uh to be able to release this uh way quicker than it used to be the case with our previous uh vendor. Yeah. Um one side quest we also had to fight from a perspective of uh let's say data security and PII data security is the topic of guardrails and PII detection.

[00:20:33]Uh in this picture you can also see um an icon for guardrails uh which we are uh leveraging to make sure that um our assist doesn't I don't know answer with u let's say advices how to build a bomb or things like that. So uh in order to uh secure that properly um what we also will talk about a little bit later is um a dedicated um model we also have deployed which will allow us to generate DQL queries uh way better than we did previously. Um yeah also the new search in our official documentation has been revamped and is uh leveraging latest advances in vector databases which we also consume uh from AWS providers.

[00:21:25]Yeah. So all in all many pieces had to be pulled together in a scalable uh AI API infrastructure uh which grows with the capabilities of the vendor and which can grow uh with the uh development of our agent harness as well.

[00:21:42]>> Yeah, I think um we do understand that data sovereignty and privacy are like the major concerns that our customers have and their customers have. So thinking of it on an earlier basis and making sure that we are following all the guidelines in in that particular aspect is something which is big to Dinatrace and being transparent about it and the fact that we are able to scale these services in a manner that it suits our customers. That thought process I think is is very nice Thomas. So thank you so much for shining a light on that and making us aware of the fact that we do all this in the background. Maybe we don't talk about it. Maybe we don't put out so much information out there. But then there is a thought process that goes in and ultimately we are limited with the infrastructure that our partners have and there is a lot of tweaking that goes in the place for us to be able to get those things working.

[00:22:38]So now we will take a little um we'll keep um rocking on and we have our next segment which is going to be skills. So there has been introduction of skills.

[00:22:49]We announced this where the foundation models always know that and then there is a lot of additional information that was provided in order to add those skills into dino trace assist. So before people like take a picture of a generic chatbot or a FAQ agent, can you explain what skills actually are and what makes them more than just pointing the model at a documentation? So it's not just a documentation rag approach but what skills are and what makes Dynamra SS different. Mhm.

[00:23:23]Um yeah. So if you Yeah. I mean I I chose um this picture here because I think that's the let's say the secret behind uh skills as proposed by entropic. I don't know how many months or maybe a year ago or something like that. And that's kind of the let's say this progressive disclosure flow what skills enable. It's that the mental thing is like um this cons uh let's view the skills the a whole set of skills like a book. Each chapter in the book is one skill and the headline of the chapra is written in the table of content and basically when the bot is starting up what he always has as a let's say post-it note on it on the conversation screen is the table of content and then and then when the user comes in with a certain question uh then it would basically relate the user's question with the uh with the with the let's a abstract and the title of the chapter and would quickly skim that through and then decide okay it fits to chapter 3 and then it would go there and read the relevant parts uh of chapter 3 and uh after examining uh let's say the the extended abstract uh of that chapter so here in in depicted with the skill MD file then it would see okay in the extended chapter this chapter talks about nanced detail is about in our case for example how to properly formulate the DQL query or how to properly uh use our documents API or how to properly um um analyze uh deterministically found Davis problems and then after having pulled all this information together it will start actioning on it so it will then uh use tools to grab data from Grail it will then use tool to grab data from I don't know from our document store pull this together onto his uh cheat sheet, compares it with the text which is provided by the skills book, let's put it that way, and the actual data and then by the power of the new brain can make up uh conclusions and decisions and gives the answer uh back to the user. So it's like let's say structured way how you find information in a book from the table of content to the extended abstract of a chapter and then going into the details of the chapters. That's this progressive disclosure flow which is enabled uh by by skills.

[00:26:01]>> Okay. So we have done that for dinet trace assist. Um now can you talk a little bit about before we had the skills what were the problems that users were running into and what was the like specific kind of questions or scenarios that Dino trace assist kept falling short of which got solved because of using skills?

[00:26:22]>> Yeah. So what we basically previously relied on uh mostly was our official documentation and this was not um that the skills we have developed now are way more use case focused and uh let's say stripped down to the uh stripped down to the mental model uh of an LLM and not to the mental model uh which is behind our userfacing documentation. which is much broader, right? And the search process which typic is typically used for let's say a huge a large scale documentation website as ours is semantic is the semantic search. You typically come in with a question the ve the question gets transformed into some numerical uh entity such an embedding vector which can then be used to find relevant parts and pieces uh in this documentation. But the um LLM doesn't have a dedicated way to say now I want to read chapter 15.

[00:27:25]Yeah, it can just query this knowledge base. And this is the uh this is in contrast to what skills provided with the skills. Actually the LLM can now say I want to read page number 29 because I know uh there is the relevant information because the chapter the the chapter of content uh said uh the table of content said so. And uh with this um with this more precise way of information retrieval um the answers are getting better because it can steer its attention to exactly the correct pages in the book so to say. Yeah, that's the from a mental model that's the difference between the a large scale semantic search approach rag approach uh which were the dominant uh uh dominant technique we used before we had skills and now uh compared to uh the skills uh we have right now.

[00:28:22]>> Okay. So I think it just means that because there is so dedicated amount of skills it's like a it's very well well definfined in terms of that table of content where do you go to find that information it's easily available it easily retrievable and then being able to resolve on it is so much easier for the agent itself and then it can get you the accurate response in a quicker and a much detailed way rather than going through the entire documentation every single time.

[00:28:54]>> That is great. Thank you so much for walking us through the skills, what it does and what it changes for assist. Um I want to move a little bit on the product development angle and then talk about like this whole knowledge base that was created at Dinatrace. It had to be done at scale because a lot of the times what happens is people say that Dino trace is an extensive platform.

[00:29:16]There are a lot of different pockets of information which are available everywhere. So doing this at scale kind of sounds hard like how did you decide on what gets put in the skills, how to structure it and how do you keep it up over time?

[00:29:34]>> So there are quite a few angles to this Safia and I think um on the one hand you know we have a couple of core system skills that we maintain at the moment.

[00:29:44]you know these are the more general generic skills that we have available that assist will always load. But then one of the things that is um super important to us especially from you know how we look at building out our entire um agentic framework is the fact that we need to be able to offload um some of the domain knowledge and the domain expertise to those teams who actually are responsible for that. So um you know in one of those earlier slides I actually said there like we took all of the knowhow that lived in the mind of our experts um and now it's actually in the product and if we do um you know move on to to the next slide I think um as one of our you know core skill contributors vulf can actually demo exactly what we mean with this in terms of bringing that domain expertise directly into the assist experience and Then after that I can give you a little bit more info um about how we are extending that even further.

[00:30:48]>> Yeah. So um for for many years um we we had actually um a dynamic forecasting possibility in in the dino trace platform. So everybody, every user uh could have uh created a a notebook or a dashboard and uh do a forecast on on the past um signal information or observability information. Um and that's done fully automatic. Uh the training of the forecast model is is done dynamically based on the filters and the queries that that the user provided. But now let me show you what Thomas explained before. It sounded very complex in terms of what a skill does and how it's how it's built. But now let me show uh what the combination of our forecasting tool in combination with the predictive observability uh AI skill uh gives you in terms of uh power and how low the entry barrier actually is. So let's see here a list of the front end applications and uh I prepared a question um that I find super interesting um and according to that question I will uh explain what actually the skill um provides and what what the tool actually provides. So my question is I challenge uh Dino trace assist uh to give me an uh an overview about the top three front-end applications in terms of user traffic uh and to to predict uh that user traffic of those top three front-end applications uh into the next week. So this is a a question that is twofold. So uh the entropic model first needs to split uh the my request into uh several uh tasks uh getting the information about uh what are the what I mean with the top three front- end applications um and then understanding um what my question is about. So it basically loads our prepared skills for what is a front end uh what is the the detail about front end uh and real user monitoring and so on and then it loads my own skill the predictive analytic skill uh and uh then understands the fine details about how to properly do forecasting and you already see here the the first grail queries are are finished. It figured out what the three front end the topmost front end applications are. That's pretty quickly quicker than than I can talk about it actually. And you see that the ondemand forecast uh tool training was actually done in parallel here. One model for each individual front end application.

[00:33:53]Um and um then it actually comes back with the forecast for the next week uh for the next seven days for those top three front- end applications. And it also gives us a nice uh overview about where action is um necessary and where no action is is recommended and what's the confidence range of of the trained forecast model. So and what the skill actually uh provides on top of the forecasting tool and the forecasting model is that it provides the data science insight on how what what are what are the how-tos uh what are the best practices what are the forecast horizons uh that should be used what is the granularity of the data that should be fetched all that is fed into the model additionally uh the forecast would also work without the skill, but it's much smoother and much more uh educated than than without the skill, right? And you see here this amazing result where you get immediate uh access to any kind of forecast no matter if it's about a log pattern uh front end application traffic, service traffic or problems uh that are detected.

[00:35:19]So if I ask it who's going to win the FIFA World Cup, it's not going to forecast that.

[00:35:23]>> Yeah, [laughter] my prediction was that Austria is coming further. But you know, [laughter] >> oh, too bad. [snorts] It's going to get into this guard rails and then it's going to be like, nope, sorry, you're blacklisted. I'm not [laughter] questions now.

[00:35:41]>> So that is awesome, U Wolf Gang that we can do forecasting. I know that this has been like a major question that our customers ask us like getting into the whole predictive things rather than being into the firefighting mode. But then having a skill that can do just that and the fact that the assist agent is capable of or the operator is capable of picking up on those skills and implement them as and when needed. So we can certainly see the difference between moving from that um generative AI chatbot mode into the agentic AI mode and what it provides us. So thank you for walking us through that. And then now over to you Gabby. I think the whole thing is that it's not just the skills that you created and the skills that we have uploaded into Dinetrace but our customers can also create custom skills.

[00:36:30]So let's talk about that.

[00:36:32]>> Yeah. So that is actually coming up um a bit later this quarter and I did want to just give a sneak peek of this because it is one of the top questions that I do get from customers at the moment. So in the same way that we have our core selection of system skills all of our apps can now provide their domain knowledge in the exact same way that Vulong provided the predictive analytics um skill all of our apps will be able to do the same. And then in addition to that because we know that what we provide is really just the baseline.

[00:37:02]Every single customer does things a little bit differently. They have some specific um you know things that are special about their environment, something that's special about the teams, the ownership, the tagging, the data, anything like that. And then what will happen is every single customer will be able to upload their own custom skills which of course will remain bound within their tenant. And then every single time that a user asks a question, all of those custom skills will be loaded as well in exactly the same way that Thomas explained earlier. And that way we are now fusing our domain knowledge with customers specific requirements and also specific guidance that they have um available to the AI about how they actually do observability within their own organization.

[00:37:51]Yeah, I think that is very much important because it's not just every customer but also every industry, every vertical, every state is now having their own things which is like I want these policies on my data. I want to do certain amount of data transformation even before I send it to the LLMs and hopefully that skills can augment all of that and also the tribal knowledge that the teams have about their applications and if they can put it out in the word doc or in the skills then they do know that that skill will be loaded and the LLM will actually be looking at that while it is providing responses. So you said a little bit later towards the end of the year. So I do understand that.

[00:38:32]Okay, it's coming. We get that.

[00:38:34][laughter] >> Okay. So um the third thing that we announced is the NL to DQL which is something we moved from the rag model into a very fine-tuned version of Dinetrace um AI which is now being able to like when you ask a natural language question it translates that into the DQL. So moving from that rack based approach to a purpose-built fine-tuned model on hundreds and thousands of real DQL examples for someone who has not followed this space or this journey why is it such a meaningful step and what made you think about doing that?

[00:39:16]>> Um yeah if I can I can take that. So um if you think about it how it was done previously it was basically a little bit like I don't know uh teaching a complete new hire who had previously worked I don't know at the company which only knew uh SQL coming into tin uh and then explaining him how to write the quail query and explaining him the data model in grail and each time from scratch um Because back then when we had the first incarnation of that uh there was no let's say there was not a big stock of uh DQL examples out there in the wild. TQL was our proprietary query language and all the available foundational models uh only had a let's say uh a teeny tiny clue about the quil and so what we needed to do at the end is uh we needed to augment the user input uh each time the user had a question to augment it with the semant with a with a big part uh of context about the DQL syntax and semantics and a big part on what's uh the grail data context. Yeah. And this sounds a little bit like uh wasting time and uh wasting yeah uh yeah basically wasting time and wasting money. And because this quail syntax and semantic change is obviously way slower than the quail data context or the the yeah data context. And that was the reason why we're thinking okay let's bake this DQL syntax and semantics uh into a dedicated brain. And that's where the finetuning comes in. And our research lab colleagues uh did that. And a very major part of that was um curating a data set of meaningful pairs of natural language questions and corresponding powerful DQL queries in a way uh that it can be fed into a training process into a fine tuning process uh for our uh Dino trace DQL expert model and the so to say the data curation part uh is a very critical uh thing which is continuously ongoing. We have a lot of data quality monitors on those training data sets which tell us how well each feature of DQL is covered, how many queries for each category we have um and what not and when new features in trail are incoming uh we would automatically pick them up. To be very specific, we only use Dino Trace internal data from our dev tenants. So we only see a TQL queries which are written by Dinatrace uh employees. None of your customer data is used for that training process. Uh that's very important to state. Um so we uh really make sure that none of your data uh leaks your environment and yeah so there's still a lot of uh stuff there which we still need because what the expert cannot know also it gets to know best practices already because Dino uh the quil experts are running the quil queries and they get into the finetuning of the model but if you as a customer have your own way of inesting I don't know are your logs or dedicated custom metrics or dedicated custom events and spans and what not. This cannot be known beforehand because as I said we don't train on customer data. So we now baked the static part of the DQL syntax and common best practice knowledge into the uh DQL expert model but still the customer specific part cannot be there and that's the reason why there is still uh a few boxes on top which take care of uh bringing the customs specific uh how we call it environment aware um aspect um uh into the in context learning And so the expert model will now basically merge your custom data with uh the best practices and the static part of semantic and syntax knowledge and is way is way better in generating those DQL queries and that it is actually way better is shown on the next slide [snorts] and there our research lab uh did an amazing job of comparing it. I think we did that somewhere last somewhere in January or or December. Uh models are popping up that quick. Um and measured the quality of um of the new DQL expert model uh against a full rack based approach against OPUS 46 and GPT43 uh 4 4.3. Yes. and on a on a test set of I don't know exactly a few thousands of uh natural language to decoil pairs uh this dino trace model uh showed superior uh performance there. So and this uh com combined with uh even shorter run times and uh uh more token efficiency uh this gives now a better experience in the let's say non chat interface where you just put in a question and get out a DQL uh query.

[00:44:57]>> Okay, this is perfect. But um u what I want to understand is that how will the user see a difference on his day-to-day like is there a concrete example that you have with a query that maybe used to come back um incomplete in the previous approach and now with the new uh fine-tuned model it just magically works let's say >> yeah for example when you look at this chart for example Smartscape used to use the new Smart cape which is incoming was not picked up very well uh with the old model. All the uh entity queries and things like that was not picked up uh that well with the old model and these are all kinds of um questions when you have asked about those kinds of things which now make uh work way smarter with the new model.

[00:45:51]>> Okay, sounds good. Um, so as you mentioned that we needed like hundreds of thousands of DQL examples and this data came in all from our internal development environments and we did not use customer data and I want to make sure that we tie that together to the fact that we are having training data at scale but we're not using our customer data in order to do the training.

[00:46:20]Yeah, that's incredibly important and it's something that um does come up over and over again. So, I will reiterate that. So, all of these, you know, the hundreds and thousands of examples that that we have, these were all based on our own internal proprietary examples, you know, so we do not touch customer data. We do not use anything from a customer environment for fine-tuning or training. We are incredibly um like we have a very strict policy about that internally you know on the product side.

[00:46:49]So that is something just to like completely set aside like any concerns that customers might have. You know we do guarantee that we do not do any fine-tuning or training of LLM based on customer data.

[00:47:02]That is very important and thank you for reiterating on that and making sure that that is ex that is clear for our customers that their data is safe and we are not sending anything out. And again Thomas that whole picture where you show that even though we do have a specified model the customer data is still a rack based approach because every customer has their own data and for every time they access something that gets appended on to the query. So moving on very quickly. Um I do know that the last thing that we had was like a very um streamlined interface which was like a side byside chat. So for the people that are looking at it an interface change interface change looks very very simple on the surface but moving from a model to a side panel I'm suspecting that it is just more about like a simple redesigning of the layout. um what was frustrating our users and what does the side panel actually unlock in terms of product capabilities?

[00:48:04]>> Yeah, so um honestly this is something that we had in development for months before because we were testing it out and we were playing around with this and honestly we enjoyed the sidebyside mode like so much that we didn't want to hang on to the modal experience for too much longer. And the thing is with the sidebyside mode the modal version actually let me cover that first. So the modal version used to pop up. It used to cover your app. It used to cover the screen in the background. But now what you have is we have moved this to the side and as you can see over here you know we have it on the side. Um and then it means that you can have your your app or you know your canvas on the right hand side and it means that you can multitask so much better. So what has completely changed is this is no longer just like this point solution. in this chatbot, it ends up feeling a lot more intuitive and a lot more integrated. So, it really has become like this thinking partner that's along for the ride that helps you with everything that you are busy doing and it helps you um completely navigate um investigate and analyze in a completely new way. So, it's a lot more intuitive and it is actually a lot more um let's say it's a lot more gripping like you you know it's a lot more engaging to to have the side by side mode because it is something that allows you to really focus on um on multiple aspects at the same time.

[00:49:28]>> That is true. It offers a lot more productivity and then it doesn't take up the whole screen space because then you can do a multiple things at one time.

[00:49:37]So, that is awesome. And um that makes a lot of sense. And for the sake of time, I'll move into agentic workflows like um we do have um if you are still using the generative you can enable agentic and how to do it is in our docs. It's available here. You can take a look at it. But then I want to quickly move towards not just what was announced but also what is coming over. So, agentic workflows for real AIdriven automation is something that we have upcoming and I want Wolf Gang to kind of take us through what does it means in practice and what make what it what what are the things or the art of the possible that it unrolls which is not possible today.

[00:50:21]>> Yeah. So, um, AIdriven automation, uh, it's one of my favorite topics and it's it's like a rabbit hole because, uh, by using it, uh, and the power that that Thomas and Gabby showed, uh, before and baking it into an automation, um, framework like, uh, Dino trace workflows, uh, it unlocks so much powerful use cases. So you you find new use cases every day by by just playing around with it and and identifying what what it actually unlocks uh in terms of automation.

[00:50:57]Um let me uh quickly give you um a live demo of that feature that is uh currently in in preview and that will be available um in the next couple of weeks. [snorts] So while we switch to your screen, there was a question that came up in chat which was like >> looping through workflow actions. Is that going to be possible? And then the other one was can you on the fly change the model for workflow actions? If you want to just talk about that during your demo, we'll appreciate that.

[00:51:31]>> Yes. So um one of the topics that we plan uh to introduce in the future is especially for workflow and uh uh automationdriven um uh AI workflows that you can uh select your uh that you can select the model yourself. So you can switch between t-shirt size models uh like haiku sonet or oppus and decide on the use case that you plan to implement with the workflow. Um but let me first show you what what a workflow uh is or what an agentic workflow is. And when you open uh Dino trace workflows, it's really the action part, the act part uh on top of your dino trace tenant. So this is where you act on any kind of situation on on a scheduled um uh trigger or on on a problem or an event.

[00:52:27]It really or on demand. It really depends on your use case.

[00:52:31]And uh we we are jumping right into uh readym made u workflow templates here uh where we already collected our own um use cases and provide you with a catalog of readym made workflow templates that you can simply enable and there are some some useful examples in in there that you can uh right away start uh to build your automation on top of the agentic framework. network in your dino trace tenant like the alert reduction agent or database operations agent or a security insights uh agent and so on. But let me show you how the creation and the work with an agentic uh workflow actually looks like and how easy it is and how low the entry barrier is. Uh one thing to note because there was the question before uh in the chat is and it's a very important aspect here everything uh works with the regular uh Dino trace platform uh permission policies. So in this case I'm running the workflow the automation with my own user as an actor but typically I wouldn't do that but I would switch to a service user and for my S sur agent I already prepared a readym made uh S sur agent service user that has exactly the right permission set in order to access only the data in the Grail data lake uh that I want my uh AI automation uh to leverage.

[00:54:09]So this is really the way how you uh can specify and um the access policy for the individual AI automation.

[00:54:18]Uh so in in this case probably I would like to uh trigger on demand and I can select here a prompt a genti and you see here that I can start off by writing the prompt. I can use uh ginger templating here. Uh I can add additional context and instructions and I can allow or disallow uh the uh workflow the agentic uh action here uh certain sets of uh tools here or the interaction with other agents. Of course, I can also specify if I want the output format and I can also force it to use a specific conversation ID, which means I can even continue a conversation that a previous workflow trigger uh did start, right?

[00:55:12]And and for example, if I have an incident and I trigger multiple times uh or on a problem, uh I can continue the conversation.

[00:55:22]And for that purpose uh I I did already create one uh workflow here that I called SR agent that I will also add to our um agentic workflow uh template catalog. So you will have that ready uh in end of August or something in that timeline. But let's let's see what this um SR agent does. It's uh triggering on a problem. I select the the uh filter on the on the problem to not trigger on on every problem but just to select the subset. Then I call the S sur agent here by specifying some prompt that I provided uh and how I expect the root cause analyszis to be happening.

[00:56:11]uh I give it some additional instructions uh and I limit the access to the tools in order to speed it up a bit. Um and then I use one uh readym made uh action that you can find in the problems app which is the uh annotation and commenting action that you can use in order to link and push back the result of the agent to the calling. uh problem.

[00:56:41]You see here uh the changer template here uses the problem ID, right? And as a description um as the comment basically I I trigger it and I push it back uh onto the calling and the triggering problem. When I run this uh I will show you how it looks like in reality. Um it uh triggers on incoming problems. Uh and uh if we look at one of those problems, you see um that the SRE agent service user did run uh the uh agentic workflow and you see the triggering the successful triggering here.

[00:57:25]uh in the automation overview also with a time stamp you see the the deterministically collected visual resolution path and all the events that were uh collected and denoised here and we see the agentic result of the analyzis. So the deep agentic analyszis actually comes back with a pretty deep root cause analyzis automatically uh that is directly persisted onto the problem itself and it surfaces unknown unknowns. It crawls through the log lines uh it finds error logs and then even comes back with recommendations about what I can do in order to mitigate this problem. Right? And here it basically comes back with a suggestion to change the queue uh of this GMS uh handler um and and to mitigate this uh issue. And this really dramatically speeds up meantime to repair uh by directly triggering and automating uh on top of uh detected problems and directly persisting back the the result onto the problem itself. And as I said, it's not limited to um um our H8 framework here, but you can also comment yourself or let your own um agent comment and and have a conversation basically on top of this problem. So it's pretty open and it's extremely powerful.

[00:59:02]>> Yeah. So that this is awesome, Wolf Gang. um having the capability to run these workflows which is the action item that what to do in case of certain problems and then tying back together all the way into the problem and the conversation ID to make sure that you can pick up where you left off. All of these are very uh useful features and powerful ones. We'll talk about them later at some um once we are more closer to announcing it. I greatly want to appreciate all of you for taking the time out and for Gabby for coming always prepared with all of the answers to all the hard questions that I ask and making it a very transparent process for our users who are actually using all of the incredible products that you guys put out together. So, thank you so much for doing this with me.

[00:59:52]>> Absolutely. It's always a pleasure to be on with you Safia >> and then we'll respond to all the comments, all the questions that came through. will go through and answer them on LinkedIn just so that you have all of the answers to the questions that were asked.

[01:00:09]>> Thank you.

[01:00:16]>> [music]