What's New in Playwright AI: Agent Features You Need to Know

[00:00:00]All right, let's kick things off.

[00:00:02]Agentic testing with Playright. Give your coding agents a real browser to watch uh and write and fix your uh playright tests. Before we get started, a little bit of housekeeping. You will receive a recording um of this entire session after the webinar. I usually edit it tomorrow morning. So, it will then uh hit your inbox and I might also share the slides if you're interested in that. And we already tested the chat. If you have any questions, comments, feedback, please drop it in the chat. I have the chat window here on my second screen. And so that I'm not here sitting alone in my office talking to my computer. I always appreciate comments, questions, uh, and if the session is as interactive as possible, that would be a lot of fun for me. Let me quickly introduce myself. I'm Stefan. I work here in Czechlist developer relations team for quite a while. I'm also a Google developer expert in web technology and I am a playright ambassador and I know that we want to talk about playright and AI here quite a little bit but we're still in a Czechly webinar. So if you're new to Czechly let me just give you a brief introduction of what Czechly is what you can do with Czechly and why we're doing playright webinars. So first of all we have 2026 and the current state of software development is pretty much this.

[00:01:21]AI is writing code faster than it can be tested or monitored. And this leads kind of to a quality gap, to a confidence gap because we're all shipping faster to production. Our agents are shipping faster to production. And it's very hard to keep track and also monitor all these new features, all these new things that we are pushing out constantly. And it's very tough when uh to ship good software and good quality when this gap between covered software and the software that we ship is just widening. And that is just what is happening for a lot of companies. There's a lot of software developers out there right now. So, we at Czechly, we believe that your monitoring should live in code. And guess what? When your agents are writing your features, your new um your new software, your agent can agents can also join the party and also write your entire monitoring too. In Czechley, everything that you can do in Czechley's monitoring platform, let it be an alert channel, monitors, play rate checks, browser checks, um even status pages, everything lives in your repository so that you can update and deploy your monitoring in the same go as you do your production code. And this makes it a perfect fit for the new world so that you can ship your new features to production and update and keep the coverage, the monitoring coverage high in the same time. And we honestly believe that playright is a beautiful software. So you can take Czechly and run your existing playright tests in the Czechly cloud to use them for uh global production monitoring. How does it work?

[00:02:52]If you have your monitoring defined in your repository, you can use the Czechly CLI and you can test things out using npx checkly test. This will take your existing monitoring and will run it on a test session in the Czechly cloud. And once you deploy to production and once you're happy with your future monitoring, you can call npx checkly deploy to turn on your production monitoring to be on the safe side and be the first one to know when something is cooking, when something is broken in production. And because we have first class playright support, we run playrite pretty much at scale and because we think that playright is the best tool out there and we are not the only ones thinking that, right? If you consider the playright test weekly download numbers on npm, this is kind of like a hockey stick. It's only not a hockey stick because I w w w w w w w w w w w w w w w w w w w w wedened the graph a little bit. But the npm downloads um weekly npm downloads of playright tests uh grew 4x over the last year and there's a single reason for that. it is that playright is the default choice for the agents out there and that is pretty pretty pretty nice I think because agents are pretty good at using and configuring playrate so if you're new to this whole AI world playrate world you might ask yourself the question first why should a an agent use playrate when you consider end to- end testing and kind of writing end to-end tests in playrate you really have to understand that an agent that cannot open a browser is really just guessing and honestly I have been there right you can feed your coding agents your cloud code your codeex whatever you're using all the source code in the world it will probably understand the components the code that you're running but if you're not really running the target environment the browser it is really just guessing and flying blind and I've tried this I fed claude a lot of lot of source code say hey can you create tests for this please consider all the things but at the end of the day you really have to use the environment that you're shipping to provide and to create good end-to-end test coverage.

[00:04:56]So if you give your agents a real browser, this whole guessing guessing stops and the generation of or the automatic generation of end to end test just becomes an entire new story. So let me show you how this actually could look like when you connect your agent to playright's AI tooling. So what you see here is a mocked session that I did a little bit earlier. It tells um the agent to open a website. Then it navigates there and then it takes a page snapshot. And that looks fairly straightforward. But check this out.

[00:05:28]Whenever you communicate with the player AI tooling, this is what you get in as a response. So a little bit of code code that the agent ran and some pointers to additional resources. And then the agent can always access what is called a page snapshot. That is not a DOM snapshot.

[00:05:44]That is not HTML. This is pretty much the accessibility tree of the browser while it was on this page. And these two ingredients make it perfect to generate or use uh agents for play rate automation because when we look for example at the the output of results, the point that there's always a little snippet in there that tells the agent what code has been run or what code would have been run makes it very easy to repurpose all these context all these all this text that is already sitting in the conversation to then create uh new playright test cases. And then when you look at uh the page snapshots that the AI tooling provide, this is perfect to build up tests because it includes all the information to for example create meaningful and semantically correct uh locators. My favorite here is get by role. So when this information is all in the conversation with your AI agent, well, it is a pretty easy task to create player tests from that. So with that if you give an agent a live session, a live browser session, it pretty much behaves like you do and how I do right because then it will open a page it will look around okay how can we interact with things then it will click on things then we'll check what is going on and this is pretty much behavior that you want to have when you want to task your agent with end to end test generation.

[00:07:06]So the question excuse [clears throat] me so the question is how can an agent use playright? I already showed you this little demo session here. So the question is how do you connect? And if you are using AI tools, I mean we are now six months in the AI revolution.

[00:07:20]There are basically two tools that you could use. You could use MCP or you could use a CLI. Which one are you using right now? Uh when you look at the npm download statistics of these two tools provided by the playright ecosystem, we see that the playright MCP server is downloaded six million times per week.

[00:07:42]And then we have the kind of new playright CLI which is not like the playright command that you use to run your tests. It's a different tool. It is right now at 500,000 weekly downloads.

[00:07:55]So the question is when should you use which? Do you use both right now? Do you use only the MCP? Do you use only the CLI? And the answer to this or what I at least believed usually is that I want to use the CLI, right? Because it is more token efficient, right? Because MCP can be very token hungry. And I think a lot of people believes that. So let me just enter the first demo here and show you how actually the tooling evolved around MCP and around uh CLI usage.

[00:08:28]So what we see here is three terminal windows. They are all in a playright token demo. And I have a little shell script here that starts cloud code and different configuration settings. So we have here a barebones cloud code that is pretty much nothing is installed. No cloud MD, no additional tooling, nothing. This is barebones um cloud when you run it for the first time. So here we have then um cloud code with the connected playright MCP server. And here we have cloud code with the connected playright CLI. And we can now have a look at what is going on here in the context window because that is what everybody's worried about. So when we check this pretty bare bones cloud code installation here, we see that right now it has already without having done anything it is already using 22k tokens.

[00:09:16]That's 2% of the entire um conversation window or context window. When we now on the other hand check out the MP and M MCP server, we will see that we see a little bit of different numbers. First of all, we have a long list here of tool definitions that are all coming from the playright MCP server. And we check the use tokens here. We see that we're already at 36K.

[00:09:39]And the majority comes of additional tokens comes here from the MCP tools. So when cloud starts up, it has to load all these tool definitions to understand how it can use this MCP server. So when we now on the other hand try the CLI setup which also has a skill installed teaching the uh cloud code how to use the player CLI. We see that we're here pretty much at also 22K and the only overhead coming in coming in here is 30 additional tokens coming from the playright CLI skill if you haven't used skills. So skills are basically markdown files that live in your project and they are have two sections. So on the top um they have like a header definition that tells cloud code or your harness, hey this is what this skill file includes.

[00:10:27]It gives a name and a description and only this makes it into your context window when you kick off your agent and this is why this is so small here. So generally you could now say that using a CLI with a paired skill file is the most performant way to do right but we will look at this in the in a second. From here on on, we could now use the MCP or the CLI server and we really could just give our um our agent some instructions like, "Hey, please use Playright to open checkleyhq.com."

[00:11:04]So what it now does, it communicates with the MCP server. CLI would pretty much work the same. And it will now open a browser window. Here we go. Here we have the browser window. And when we now check the output, we should see here what I already showed you. Here we have it. So it would if we would like to do that programmatically, it would be a page go to call. And this is how you pretty much can control um playright or browsers from within your agent interface. But let me tell you a catch here. So when we look at these, so what I just showed you is pretty much what I believed since la or um last week still. So when we have now for example the MCP setup and the skill CLI setup and we inspect the outputs, we will see that the outputs pretty much look the same, right? When we look at these, this almost looks like um an I an an complete copy so that it's maybe using the same tools um below, which it's actually doing. And the point here is when you give your agent a more advanced task like this one, please open a URL, search for something, click the first product, add the product to the cart, validate the card, show. So this is already kind of like a test case, right?

[00:12:25]When we then check the context window resulting after both these tools have succeeded doing the task, we will see that the messages spent. So I did that actually yesterday and I measured that quite a few times that the messages spent between MCP and CLI are almost the same. Then we only have the difference of MCP tools that add to the context window. And then we see this difference of roughly five sixk which kind of means that the CLI approach would be more po per performant would be more token efficient. But here's the catch. This is this information is kind of outdated because when I run my cloud code right now with the play rate MCP server enabled not in my demo scenario but with my normal setup you will see that all these tools that come in from the play MCP server are actually eating no tokens because at least cloud code I'm pretty sure the others are doing that too um found a way to optimize MCP tool usage and how to um keep the context window tidy. and clean. So this whole idea that MCP servers are eating so much tokens, I think this isn't really true anymore.

[00:13:36]And there were multiple things that happened over the last 6 months, 9 months in the entire AI ecosystem that kind of led to this parody of MCP and CLI. So first of all, there was a spec change of MCP and that is already one year old. So the MCP specification initially said that all the resources that all the resources need to uh be inlined in the responses. This was changed and the MCP server can now also um inline links or paths to other resources which means that for example if playright um generates a snapshot of a certain page this doesn't need to be in the conversation but the MCP server and the CLI just point to a certain file that's sitting on disk so that all this information is not um entering the conversation and is not kind of leading to context rod. Then we have of course the introduction of agent skills which is this mechanism of skill files living in on your machine in your project that are only pulled in lazily and uh then there was the change or the idea of code execution with MCP. So basically what cloud code and all the other harnesses are doing is they're checking all the tools that are available from an MCP server and then they're doing some magic and they also allow them to lazily only bring in the tools calls with a little bit of code execution so that you don't have to have all this uh tool call definitions in your conversation and in the case of playright this really means that MCP and CLI are just two interfaces to the same tool and there's no real um benefit of using one over the other at least not when it comes to uh considering the token spend and the context window. And I think this is quite surprising because u I always thought that the CLI is pretty much uh that we should use the CLI for saving tokens and being more efficient.

[00:15:31]So how hard is it then to gen generate tests uh to generate tests, debug tests? Um let's see how we can use this tooling to go uh for our end to- end test scenario.

[00:15:44]So AI has a question and says what about play CLI additional capabilities to open multiple sessions.

[00:15:51]I will show these and this is uh definitely okay has a question about things I will going to show you later and these are the points where I would use the CLI. Great question though. So let's jump into my um editor here. So what we're going to do is we will uh write or generate some test cases for a site that I have already running on local host. So what we see here is localhost 3000. It's a standard e-commerce shop. So you can navigate around, you can add products to the cart, you can do all the good stuff, right? So this is a functional website that is running right now on localhost.

[00:16:29]And if we now would like to generate a test for it, it is literally a single prompt. And I think this is fascinating all the time when I see it and when I execute it. So please use the playright CLI. Um you will see that I still favor the uh CLI for quite a little bit.

[00:16:44]Please use the play rate CLI. Open local host 3000 and headed mode. Uh I want to perform the following steps and please create a new um test at deos spec demo specs. What we want to do is we want to navigate to the URL. We want to search for liquid. We want to click the first product. We want the add the product to the cart. validate the correct price and close the card. Let's go. And with that um cloud code in this scenario will now use first of all load the playright CLI skill to understand how the playright CLI works. And from there on it will now look at another skill file that is covering the test generation and it looks at some existing specs files and now we should see a browser spinning up and here we go. And we'll now just poke around and kind of figure out what is available, what is accessible, and how can we uh kind of uh perform the task first of all. But then all this information will enter the context window that generating a test um will be pretty straightforward task for the LLM.

[00:17:48]So right now we already made it to the product detail page, right? That looks good. Now we're adding things to the cart. Let's go. This is cool. So we can now just let it run. And when we look at this, we see that all this context is already filling up. So in just a little bit, it will figure out how to write a test case. So let's see. Um, that is not a stable selector. That is good. Already doing some quality control.

[00:18:16]It found a test ID. So you see where we're heading here. When you give your agent the proper tools, it really can go down and do the same things that I would do with codegen or by using or by knowing the application code. All right, so that looks good. I think we already have a new test file which we have here.

[00:18:35]So this looks good. I guess there's already one thing that I don't like about this. It's hard coding the um price of the products. So let's please don't hardcode the product price.

[00:18:50]Let's go. It's now already running the playright test demo spec. So I guess we will be there in a in a second. We already have here testing test case. All right, let's go.

[00:19:03]That looks good. It found another card item price test ID. So it will readjust that in just a second.

[00:19:12]And then we are already good to go with our first generated test.

[00:19:18]It's very hot here in my rooftop apartment.

[00:19:22]All right, so that looks good. So when this is now done, we can head over to the terminal and let's go into this demo tab and let's see if that actually works. So we can go npx playright test and let's just do some quality control to check if that works.

[00:19:40]Here we go.

[00:19:47]And that looks good. But let's go back into our um source code and let's see what happens when the test case is breaking. So what I will do now is I will go into the application and I will introduce a subtle bug and I will just change what is displayed in the card there. So how could we now go about this fixing this um this behavior? So let's just run um the test case again again in headit mode. Here we go. So this is all working. This is all good. But the test now should fail because we have here 951 which is definitely not the correct price.

[00:20:25]And we as humans, how would we go about this now? So we as humans, we would now call this command here, right? And we would look at the test report.

[00:20:35]Here we have it. And then we would look at this. Maybe maybe we would even look at the trace file that is included, right? So we could have a look here.

[00:20:44]None of this is really accessible to to um your agent, right? So for a while now there's this copy prompt button here in your test reports that you can always use. So when we go here and we have a look at what is included here. So if you want to hand this task over to your agent, you can just copy prompt and give it this right there's an error there.

[00:21:03]There's test source. So this is already a lot of good information, but we are kind of dealing with a very simple scenario here. What also is already available in the test report is that it's written here that there's an attachment including the same error context. So with this change when your agent runs a player test and that is failing you can also or the agent can also look into the test results and it will find the same error context here.

[00:21:27]So the agent doesn't have to open up the test report or kind of parse things. All the information that is required will be in here. But this is still not great for kind of complex scenarios or where you really have to dive into um a problem that isn't necessarily character that is misplaced like in this example. And then there is a new playright command that I think is very very cool. So let's close this here. And what you can do basically is you can use the standard playright binary and you can call npx playright chest trace open. So let's see is that correct? No, that is not correct.

[00:22:05]And then we can open the trace file here with play rate itself. And now check this out. This is a text representation of your Is my camera frozen? No. This is a text representation of the playright report and did playright tracing. And from here on on we can drill into whatever is happening in this playright uh in this playright in this play failing playright test. Right? So there's also play red trace actions and this basically gives us all the information that we usually would have in our trace file when we dive deeper.

[00:22:42]So we have here all the actions. They have a certain index. They point out the certain code. They show how long these kind of things take. And now we can dive into this if we wanted to by calling for example accessing all these snapshots that are for example here on the 13th step. So with this new trace command, your agent can basically navigate um everything that is available from the test artifacts and it can drill into the network layer if it wants to. So with this, this is just another interface to give your AI and your agent kind of a way to poke around and and debug your failing test cases without kind of inventing things, without kind of guessing things because all the information is right there in your trace file anyways. But the thing here is that you have to teach your agent how to do this. And to do that, you should also have here player trace install a new skill. Um, so I already did that. So we can have a quick look at what this looks like. So when we go here into my cloud session, we have here skills playright trace. And this file basically teaches the agent how it can dive into the um zip file, how it can understand what is going on, how it can debug like we would do in the normal place trace viewer um in play rate. And with this information, let me just show you how this works.

[00:24:02]When we now go into this test file here, we or in our LLM conversation, we could go in there and say, "Hey, the test broke. Please investigate the trace."

[00:24:10]And now we can see the magic happening because the agent will now do the same thing that we would do. So it will locate the trace file now and then it will use the um the uh trace command hopefully in a second. So it's now rerunning the test case.

[00:24:27]Let's see if it will join the session with a proper trace investigation.

[00:24:35]It loads the player trace and now it's investigating the trace. And I think this is honestly pretty cool, right?

[00:24:42]Because it's now diving deep. It looks at all the actions that failed and it will now go on and have a look um to find out what is going on there. So this is just the first tool that was added to make playrs agent experience a little bit nicer. But if you want to dive deeper or you have a little bit more complex scenarios, there's also another tool that you could use right from the CLI. So let me show you what you can do too. So when we now go back to the terminal, we could also go for the same command here. We want to run the single test. But what we want to do is we want to debug this now with the CLI. And now check this out.

[00:25:24]So now this now spins up the Playright tooling that you might know. It's like the debugger and the window that you can follow along. But now this session here runs in a debug session that allows your CLI to connect to it. So when we now run this command here, let's see. And we attach the playright CLI to the running command, we're now attached. We can kind of control the debugging session right from the terminal. So when we look at this, so it is a three. We can now control the uh debugging session. So let's check this out. We are now right here on the page code two call. And when we now execute this to the ex uh to the attached session, we are on the next play rate instruction. And we run it again, we are on the next playright instruction. And if we want to dive into the DOM state or the accessibility state, we can now call snapshot and access um all the current state of this page. Of course, I as a human, I would never ever do this because I prefer to click buttons and kind of inspect the browser window. But as an agent, that is kind of nice tool to have. So with that, we could just have another demo run here because we have the tools for our agent available to also do a normal debugging session. So when we go back to VS Code here, we could paste another one. Hey, the demo spec ts broke. Please run playrite with the debug CLI flag and debug with the playright CLI tooling.

[00:26:48]Read the CLI skill first. Just making sure that it really knows what to do.

[00:26:53]So from there on it should now start its own debugging session kind of trying to understand what is the issue and I'm really just showing here the tools and what the CLI should uh or could do um to figure out while you're dealing with a failing player test case. Um again for use cases like this one I have right now with a a character that is there too much that is probably not the right example. But for really hard examples, you will see that the agent might take all the uh tools that it can get. And you already see it attached itself to a running browser session. Now it pauses here at the failing session just to inspect the DOM there. So this just another tool that the playright agent or the playright CLI then provides so that you and your agent can debug uh what is going on.

[00:27:40]Then there's another thing that especially when you're not using playright for end to-end testing only but it when you're for example when you're vi coding or when you're creating um your products with agents uh I use the playright tooling quite a little bit to kind of um double check what is coming out there right the browser to verify that the functionality actually works and there's also another thing that you can do with the playright CLI so when we for example use the playright CLI Okay.

[00:28:10]And we open another website here, right?

[00:28:14]So let's go for checkhq.com. We can give this session a name, right? So when we now hit this up, we should have a new browser session that is not visible right now. But when we now use the show command, we can um check out and use a very nifty feature that um I'm really into. So when we now go for npx playright show, we can first of all see all the open playright sessions.

[00:28:42]But when we now specify that we want to access the feedback session with the annotate um flag. Check this out. Play will now open up a window. This is the current state of the browser session that is currently running headless. And now we can make annotations here. We can say something. What do you think about the header? We can add that and we can submit this feedback. And when we now submit this, this will go into this process here, right? And when I do that manually, that is not very exciting, right? Because we see here, all right, there was a rectangle at these positions, there's a screenshot, and there's a snapshot. But if we do the same thing and pipe it to an agent in a flow where we're building features, um I think um that is actually very very handy. So when we open up here now a new uh conversation we could have a prompt um usually you would have that in your own skill files or whatever and we can tell it hey please open local host 3000 with the player CLI let me give let me give feedback by annotating the design show annotate using the play CLI so let's see so it will now open localhost 3000 again in headless mode that looks good all Right now it should connect to it. Here we have it.

[00:30:09]And now we could go here fix the header.

[00:30:13]Right. So this just feels like very native tooling when you're going all the V coding way. So this now entered the conversation. Right. So with this way we can kind of use the playright tooling for our let's stop this because there's nothing to fix for our um agendic engineering for our building when the agent goes I very often see myself just taking screenshots and pasting them um but that makes it very explicit because there's also the accessibility snap snapshot available and that's just a very handy way to uh to get the right context into your LLM conversation and And then there's the last thing that I want to show you.

[00:30:52]So let's just go here and fix our little typo.

[00:30:56]Especially when you're dealing with agents and flows and automated ways on dealing with product development, but also your end to end testing, there's some new um video tooling that I think really rocks. And I will not show you the um the things that you can do here manually. I will just give it another prompt so that you can play around uh maybe later. And let's see how this goes. So use the playright CLI to create a video demo with captions. Navigate to localhost 3000 and add a product to the card. So now should figure out the playright skill which we have here. Then there's a video recording uh prompt or information in the playright CLI um stuff. So what we see here here we have it. So there's a new API available in playright that is called screencast right. So here we have page screencast show a chapter start the screencast.

[00:31:47]Then we have here very show chapter. So these are basically all overlays that you can put onto your playright videos or in your playright sessions and then you can generate a video um with annotations with highlights of what is actually going on. So when you then use your AI or your agent to build new features, you can just let it run and can tell it, hey, please create me a playright test first of all, but then also let me create me proof um a play rate video that shows me that this actually works and how the flow is and what is actually happening. And I think this is a very very nifty feature.

[00:32:24]Um there's the question if this video is um available for later viewing. Yes, it will hit everybody uh via email tomorrow. So, it's now writing a playright script. So, let's see where this is going. So, here we have it. Here we now have a playright script that uses the APIs. I'm not going to look at this too much. Now, it's running the code and we should have a new video in just a little bit.

[00:33:00]All right, let's see.

[00:33:04]You can do it. My computer is already flying.

[00:33:18]You can do it.

[00:33:24]If that's not going to work, I have a fallback available. Video rendered. All right, it's in my temp directory.

[00:33:37]All right, let's see if we can open that.

[00:33:42]Let's open this one and let's see what we now generated. Here we go. So we now have here some chapters some things that are happening and basically we have running playrite the executing some things and we can send this out especially after a longer kind of session that is agentdriven you can just look at this and say yes this is actually what um I wanted to do says this is awesome be from my side chevan well we already know each other um you have to remind me where we've met. I'm sorry. And yeah, this is the new video API. I think this is very very cool. So with that, uh when we check this out, um of course, Playrate has all the goodies that you would expect from end to end testing, right? All this autoing the traces, there are a lot of things that make playright stand out stand out already.

[00:34:40]But with these new debugging capabilities, um it's just really agentic first or agent first. So first of all there's context context context right and when you have an error you can use the error context you can give uh get your agent on the right path then we have the new npx playright trace command which means that when you look at the trace file you can also just give it to your agent and we'll poke around at the network poke around at the page stage and that's just another tool that the agent might choose to um to debug your failing test cases. Then we have the debugging flow which honestly blows my mind. Uh I haven't seen the agent reaching for that too often but you have to provide a little bit more for context see how it goes but just that just that this works is kind of wild in my opinion. Then we have the annotation for kind of wipe coding so that you can give feedback and then we have screencasts as proof of work if you're generating things.

[00:35:38]And these are just a few new features that reached or entered the playright ecosystem.

[00:35:46]But the only thing that I uh want to point out here if you're using playrite right now and it doesn't matter if you use playright mcp if you use playrite cli the thing is if your skills are stale or playright is not up to date you and your agent are really missing out and you're not getting all these new features.

[00:36:08]Um, Vivvec has two questions. So, let me check. Um, the first one, is it possible to capture numbers of credits and tokens consumed by agents and try to reduce the usage of tokens? Um, I think this is a general approach for your harness I or your your agent tooling. I think that should be possible. You might have missed it. I just measured it on on the way with the /context command uh in cloud code. Um so this is a way to track. I'm sure there are also automated ways to to track that. Um in my cloud subscription I rarely hit the limits. Um but I just to keep it lean try to keep it lean. And then second is it possible to create multi- aent orchestration for testimation and adding guard rates on it like planning agent can use higher model and the other one uses lower models. I'm pretty sure that is possible. Um, let me note down this question, take a screenshot and come back uh to you tomorrow with an explanation on how you can do that.

[00:37:09]All right. So, if you are familiar with the Playright ecosystem, you might now say, "Hey, but Playright ships its own agents, right?"

[00:37:20]And it does. Indeed, it does. So you can run npx player at init agents and then you can install agents for uh planning, generating and healing. And let me tell you, well I'm a little bit skeptical here. So when you install the agents um in your in your project, this is basically what you get. You get a testing planner. This is 51 lines and most of them as tool call definitions.

[00:37:46]Then we have the player test generator and then we have the player test healer.

[00:37:50]And for the agents, coming to the previous question, you can definitely define a model, right? I'm not sure if that works with skills, but I will investigate that too. So, we have here three different agents, right? And generally, I think this is a good idea, but I don't see myself reaching for them like at all. And there are a couple of reasons for them. And maybe if you're in the crowd and you have some comments on that, I would love to hear them. But first of all, the added context here is there's not a lot of information in there. Um, for my projects, what I prefer is having my own skills that teach the agent how to perform, how to do things, how to go on with things. So, I think these 50 lines are not very valuable for the things that I usually do. Then these agents use kind of like a hidden MCP server that is coming from player test. And I honestly don't really like that because it's kind of some hidden tooling that confuses me that is there and I prefer to have visible CLI tooling. And then we have 2026. If you run a recent models, in my opinion, the models are pretty good on planning healing just fine on their own. And when you have already some tests, when you have some things going on already, you might have seen that that the agent or that the element was reaching out to the other test files and it was just doing fine. And then when you look a little bit at what is going on on GitHub and the playright repository, it looks like that the investment in the agents there um is fairly thin. There were some commits on these agent, but it looks like they're just sitting there and there's not much activity. So, it feels a little bit like this was a try out of 2025. I don't know. Um, but generally, I'm not reaching for these agents. If you do, please let me know and I'm would love to I would love to um get some information.

[00:39:33]So, Arti says, "I feel that playrs agents are not maintained as well um as it was Debbie, the one who pushed it." Yeah. Yeah. Debbie left Playrate.

[00:39:42]Um but yeah generally I think if you use a powerful model the LLM will just do fine. So with an agent with a recent model and a real browser I think it can plan generate and heal very much on its own.

[00:39:56]So as we hear with check webinar you can go from agent written test to monitoring of course with an agent very very quickly too. So the way that that works is if you have everything set up, you can use an agent plus uh an agent plus playright to get the test rolling and then you can use an agent plus checkly, excuse me, to reuse what you just generated and transform it into production monitoring. So in our example here, we just did demo specs, right? But whatever you have in your player codebase, you can reuse it and turn it into production monitoring. And when your things run in checkly, what is happening then is that we when there's a failure in your player test suite running as monitoring, we use our own agent Rocky to investigate and tell you and give you some pointers already when you receive an alert that you have a production issue. So let's turn this into monitoring.

[00:40:51]So we're in the same project here. We now have one test file that we just created, right? Demo specs. That is new.

[00:40:58]So let's clear this and let's just tell our same agent here, please create a new checking monitor for the demo specs file. Add a demo tag to the new player test and deploy. Ignore existing failing monitors because I have some things going on in my account. All right, so it now reads the demo specs.

[00:41:19]It leads the player config.

[00:41:22]It should now pull in the check skill in just a second.

[00:41:31]Don't let me hang the check skill.

[00:41:37]So, in the check skills, we pretty much tell um how Here we go. Um how to how everything tailors together, right? How you can go with things. And basically what it now should happen is that it should create a new check definition file. And then it will just tailor this new check definition um to the play rate test case that we just created. And then it will call npx checkly deploy to turn the exact same test case into a production monitor. If you now wonder how a test running against local host um can run in another cloud to monitor production, I have a little bit of um plumbing going on here. So my test cases know where they are running. Um so when they're running for example in the checky environment they're going against the production environment.

[00:42:24]All right. So we now have a new check file. So what we see here this is how you can take your existing player codebase and can transform it into play monitoring. So we just specify the player config. We say that we want to use a player project a playright demo.

[00:42:39]I'm pretty sure that it now added here a new tag to target this test case. Here we have it and then it wants to deploy. Um we have some guardrails in the checklist CLI. So when an agent wants to deploy the monitoring um it will uh really try to enforce telling me uh that it really should do this. So this is what we're seeing right now. So try to deploy. We rejected this and now we can say yes please deploy. Here we are with a typo even.

[00:43:10]Okay. So, it's now deploying and with this we're pretty much closing the loop. We generated a new test case and we're now turning it into production monitoring. So, when we now go into my Czechly account, a little bit of confetti. That looks good. We can now do a refresh here. So, if you haven't seen Czechly, this is the Czechly monitoring dashboard. You see here that I have an AI check. We have some common uptime monitors. We have here failing player check for the product catalog. And here we already have the demo flow that we just created. We can now go in here. We keep track of all the results over time.

[00:43:47]We can keep track of all the errors, all the locations, all the alerts. And when we want to dive deeper, here we have the playright results. Um, if you're familiar to the playright test report, this will feel very familiar. So you will find all the good stuff. The player traces will be available. The network is available. So all the things are just at your pleasing here. So that is very cool. So with this kind of approach, we now turned on our new monitoring. But let me show you what it means when there's an alert or a failure. So here we have a product catalog that tests that is already failing for a while and it's also running in Northern Virginia right now. So let's just look into this Frankfurt test case. And whenever there's a play rate error that is happening in the Czechly cloud, what we do is we kick off a root cause analysis coming from Rocky AI. So what you see here is that we look also at the traces, we look at the code, we look at the information, we look at previous runs to figure out if something changed um in the in the results. So we provide you a lot of we provide a lot of information that you can use to kind of figure out what is going on because when you receive a playright error from Czechly right you're dealing with an issue in production but the cool thing about this is that of course we also want to be um agentic first there too right so when we're in the same project all this information that I just showed you for example is also available via the CLI right so you can access the dashboard you can go into all the check results results like this one. This is the failing one via ID. And even if we wanted now to figure get this AI analysis that is already done, all this information is available via the CLI too. So that means that we can now go in here and we could hand this task also to our agent. So let's clear this up.

[00:45:36]And this basically automates the entire flow. So it will now um load the checklist skill um access all the check information. it will discover that there is already some AI investigation going on and then it will propose maybe it will poke into the codebase maybe it will go for some more information um but this just gives your agent superpowers and will speed up the entire flow so let's see it found the failing check already and of course you probably will never ever use the CLI for doing these things but having an interface that is agentic first really makes a difference when you want to hand over all these tasks to your agent And here we see it. It now found the AI um the AI analysis. So, and I will stop that here. So, that's not changing more things. But yeah, that is closing the entire loop. So, when you get an alert at 2 a.m. I'm pretty sure you will appreciate all the help that you can get. And with that, we are at the finish line. If you remember this gap, right?

[00:46:37]Everybody's shipping software faster and faster and faster. and all these agents are pushing out code all the time and it's very hard to keep up. Well, we think that well with the testing tooling coming from playright and with the monitoring tooling coming from Czechly, you really can close this gap and you can ship very very good things. So the closing line of this webinar here today is like code is cheap. We all figuring this out but confidence is not. So you should all take the flows that you have for your software development or all the things that you you you learn with all the agents and apply them to your testing and your monitoring too. And with that we had 50 minutes and if you have any questions now is the time.

[00:47:19]Thank you all for watching.