Stop Breaking Production: Building a local validation pipeline - Martin Howlett - PSConfEU 2026

[00:00:03][music] >> So, good afternoon everyone. Uh welcome to Stop Breaking Production Building a Local Validation Pipeline.

[00:00:19]And as always, want to kick off and say a big thank you to our sponsors and our organizers. Uh without these guys, events like this wouldn't be possible.

[00:00:27]So, a big thank you.

[00:00:30]And then, just a little bit about me. Uh my name's Martin. Uh been in tech for over 20 years now. Started off as a Windows admin. Um then went to work at Rackspace. Left there as the principal engineer in the Windows automation team. And now I'm a Windows engineer at Jane Street. And uh two relevant bits from my CV for this talk is uh Rackspace, I built the PowerShell CI pipeline. And currently working on the validation pipeline at Jane Street, as well.

[00:00:59]So, Jane Street, if you've never heard of us, um we're a quantitative global trading firm. But most importantly, we absolutely love PowerShell. So, I've got some top-level figures here. And uh we do actually have 600,000 [clears throat] lines of code. I counted last week. But obviously, that's not a a metric to anything. But um we do have over 225 modules, 11,000 Pester it blocks, 55 GRM points, and 37 custom script analyzer rules.

[00:01:27]So, we love PowerShell and we use it a lot. And um if you're interested, uh last year my colleague Izzy did a talk on migrating from PowerShell 5 to 7 and created some custom script analyzer rules to help with that. So, his talk's up there if you want to check it out.

[00:01:44]So, the problem, what are we going to be discussing today?

[00:01:47]So, I think everyone here writes PowerShell, most likely. And I think every team or every person writing PowerShell wants that code to be valid.

[00:01:56]I think the problem that we have is it often lives in the wrong place. So, it might be just in your head. You might have loads of experience. You know what you're doing. Uh it might be in a PR or in a wiki checklist that you tick off every time you do code review. It could just be in your CI in configuration. So, you write your code, you commit it, you push it, and then you wait for your green ticks. Or in a very worst-case scenario, your validation's in production when you break something.

[00:02:22]And I think the big problem with that is it means that developers, i.e. us, the people writing code, and our AI or LLM agents, they can't run that pipeline during the dev cycle.

[00:02:34]So, my pitch today is that we take all of that good validation code that we have and we move it into a single reusable command. We don't leave it trapped in our CI. We don't leave it in a wiki checklist. And we don't leave it in someone's LLM prompt.

[00:02:48]>> [clears throat] >> And I think this talk is relevant to you if you don't have CI or a validation pipeline right now.

[00:02:54]What I'd encourage you to do is start local, as I'm just about to show you.

[00:02:57]You can do it in literally 20 lines of PowerShell.

[00:03:00]You can prove the checks are useful and build it out from there. And if you have an existing CI, I want to challenge you to refactor that so developers and LLMs can run that CI validation locally. So, either way, there's one validation pipeline with multiple callers.

[00:03:16]And what do those callers look like? So, in these blue boxes here, where these are the people who would want to consume a validation pipeline. So, you've got your developers, the people writing code.

[00:03:27]So, I'm sure we've all been there when we're writing a long script or a module and you've reached flow state, right?

[00:03:32]You're deep into it, you're writing away, you're being really productive.

[00:03:35]And then it completely takes you out of it if you then need to commit your code, push it, go and look at your CI configuration page, wait for it to finish. You know, that's a 2-3-4-5 minute step. And what that does is breaks your flow and it breaks that development loop. And it can be really quite jarring.

[00:03:53]So, by bringing it into a single command, you create that instant feedback to allow you to move really quickly, and it's actually a really nice experience when you do that.

[00:04:02]And of course, if we can run it locally, it means our LLM agents can also run it.

[00:04:07]So, I've got some examples to show you later, but what can happen is an LLM agent will actually iterate over the code they've just written, realize it's probably not quite to the standard that we need, they'll detect the error, and then they'll fix it. And that's because a validation pipeline is deterministic.

[00:04:23]It's always going to give you the same output. And then LLMs are pretty good, but they're not deterministic. They can randomly decide to change stuff and pull out stuff from under you. So, by having this single deterministic pipeline ensures that your code is valid.

[00:04:39]And of course, we have CI. I'm not proposing we get rid of CI. CI is great, and it should always be that last enforcement gate. It should be the last guard before your code reaches production.

[00:04:49]So, but the key difference here is that the code that I run as a developer is exactly the same validation pipeline that your um your CI runs.

[00:05:00]And how we do that is with one single validation command.

[00:05:04]So, in this case, it's Invoke-BuildValidate.

[00:05:07]And what that gives you is the same rules, the same entry point, and the same output.

[00:05:14]So, it unifies everything. So, wherever you're validating your code from, it all goes through the same process. And the end result is lots of green ticks and happy developers.

[00:05:26]So, I've been talking a lot about validation pipeline. What does that actually mean? So, the way I've broken it down is it it's roughly 10 tasks that you can run every time you want to validate your code. And because 10's, you know, it's quite a lot it's a a lot to conceptualize, I broke them down into four categories.

[00:05:44]So, the first one is code quality. So, this is going to be script analyzer and Pester. They're probably your two heavy hitters, right? If you don't have any validation now, if you take away and implement those two, you're going to be a lot further than you are now.

[00:05:59]And of course, code coverage. You get that for free with Pester, and I think it's a useful signal when used appropriately, right?

[00:06:06]And then we've got release hygiene. So, release hygiene just make sure that your module is ready to be built and pushed to the PS gallery or Artifactory or whatever new repository you're using.

[00:06:17]So, here we just make sure that the module version's been bumped and that the change log has been updated.

[00:06:22]Uh the amount of times that I've had to push back on a PR because someone's forgotten to update the change log is far too many. But since we added a validation pipeline, it checks that for you, so it removes that back and forth.

[00:06:36]And then we've got content. So, this is more your mechanical side of like your JSON and your YAML files.

[00:06:42]So, uh a long time ago, I was working on a uh very large JSON configuration, and I removed a dictionary from it, but I left a trailing comma. And in a PowerShell 5.1, that's a death sentence.

[00:06:55]You go to uh convert it from JSON, and it just dies. And it actually caused a production incident, and it was extremely embarrassing.

[00:07:02]>> [clears throat] >> So, by adding this check into my validation pipeline, I've never made that mistake again because I physically can't, right? I make sure that my JSON is loadable.

[00:07:12]And you can extend that to schemas.

[00:07:14]So, um if you have a very large complex JSON, it's really good to validate that with a schema. And then if you have a schema, you need to make sure that you're actually validating your JSON against a schema. That's quite tedious to do manually, but if you build it into your pipeline, it just happens automatically and again will catch those edge cases.

[00:07:32]And then we have our module conventions.

[00:07:35]So, in these examples, this is probably the most opinionated part, but this is something you would need to decide if you as a solo dev or in your uh repository or as your team how you want to lay it out your code. So, you may want all your functions in the PSM1, you may split them into public private folders, you may have certain naming standards where the module name needs to be in the noun, all those kind of things. And these are going to be very specific to your team, but enforcing them at this level means that if you make a mistake and you forget, you're instantly going to know about it during that dev loop when you're in that flow state and you go, "Oh, yeah, I forgot. I need to add this."

[00:08:11]And what that does is reduces your cognitive load when you're trying to review code. And I don't know about you, but with the advent of AI models, there's a lot more code to review. So, if you can have your AI agent automatically enforce these sort of standards, it means you can actually focus on the code quality and the business logic and not how the module is laid out.

[00:08:37]So, I've talked about Invoke-Build a lot.

[00:08:39]I assume most people in this room know Invoke-Build. There's been talks at PS Conf in the previous years.

[00:08:46]But [clears throat] I'll just quickly talk about what I like about it. I love the fact it's pure PowerShell, so there's no third-party dependency, doesn't have to run in the cloud, it's doesn't cost any tokens to use or run.

[00:08:58]And I like the fact that it's cross-platform. So, in my own stuff I write in my own time, I use GitHub Actions. And when you run your CI pipeline on Ubuntu, it runs in half the time. So, that costs half the price, right? And most a lot of PowerShell modules can actually be run in Ubuntu or any Linux distro, right? So, by making it cross-platform and using a cross-platform tool, you can save money that size and then it can be faster overall, stuff like that.

[00:09:27]And then I think the key thing I like about Invoke-Build is it means your engine that's running your validation exists in one place. So, at Rackspace, uh we use Jenkins, which is like a monolithic Java application that runs your pipelines, and you describe your pipelines using Groovy script. Uh but over the years, what happened is that turned into a mess. So, in order to edit our pipeline, we would have to make one change, push it, wait 5 minutes, and see if it broke, and it was really painful.

[00:09:59]So, we ripped all of that logic out, moved it into Invoke-Build, and it meant I could just run my pipeline locally and troubleshoot it there. And then when I wanted to ship it into the CI, I just copied it to Jenkins, and away we go.

[00:10:11]So, it really simplifies everything moving uh your validation logic into that same engine. And obviously, having your validation logic in one place makes a lot of sense, cuz if your CI drifts from what you're checking locally, again, you'll hit that frustration that when you push your code, something fails you didn't expect.

[00:10:32]Fantastic. So, that's enough theory. So, we're going to jump into some demos now.

[00:10:36]Uh I'm just going to briefly show you Invoke-Build, just so we have some context. Then I'm going to show you some various pipelines. We've got a very small one, and then we uh have a full pipeline. I'll show you failures, and then we're going to add a task live just to show you how easy it is. So, see how that goes. And then at the very end, uh I've got some context to give you around running it in the CI, and then finally, I'll show you quite a cool example of an LLM iterating using this local validation pipeline.

[00:11:06]Okay, good stuff. So, let's start with Invoke-Build.

[00:11:11]So, to install Invoke-Build is really simple. It's Install-Module Invoke-Build.

[00:11:15]Not much more to say there. We'll install PSResource if you're uh up to date. But the way uh Invoke-Build is that it uses uh a build file. So, when you run Invoke-Build, it looks for a dot build or a build.ps1 file in the folder that you're currently in.

[00:11:31]So, what I normally do is in all my modules, I just put a build.ps1 with my validation pipeline. Then, when I run Invoke-Build, it's going to find them and run them. So, in all these demos, we're going to be working in those build files that are called when we run Invoke-Build. [clears throat] And this is what the syntax of Invoke-Build looks like.

[00:11:50]You'll see here that it's simply Add-BuildTask. You give it a name, in this case task one.

[00:11:55]And then, the script block is what's going to execute when you run that task.

[00:11:59]So, in this case, it's just going to write to screen, this is task one.

[00:12:03]And Write-Build is just the wrapper around Write-Host essentially that Invoke-Build uses, which is why it's slightly different.

[00:12:10]And then, we've got a second task here where we again we're using red. Um sorry.

[00:12:15]Again, we're using Write-Build. We'll output in red, and it says, "This is task two."

[00:12:20]>> [snorts] >> And then, here is where the actual magic happens. So, what we can do is then combine tasks to call from another task.

[00:12:27]So, here I have a task called my pipeline. And what that's going to do is just run task one and task two.

[00:12:34]So, uh just a little note about tasks, jobs, and pipelines.

[00:12:39]So, a task I like to think of as like a bare unit of work. It's the equivalent of a function in PowerShell, right? You call it, executes a script block, does a thing.

[00:12:49]Uh the reason it's referred to as a jobs here uh is because you can run a single script block or you can run multiple tasks. So, internals of Invoke-Build just abstract that out to jobs. But, essentially, it's the same thing. And then, pipeline is just something I call it cuz pipeline sounds cooler in my head than a series of 10 tasks that validate your PowerShell code. Um so, what that means is you can have a single task that can call lots of other tasks and build you this entire task graph of stuff to execute.

[00:13:20]So, now we're going to jump over to the CLI, and I will show you this in action.

[00:13:24]So, I'm in the the folder I was just showing you that build file. So, now what I can do is run Invoke-Build task one, and what we'll see is this is task one in yellow.

[00:13:34]And then I'm going to run this again with task two, and what we'll see is this is task two in red.

[00:13:41]Excellent. So, now what I can do is run Invoke-Build my pipeline, and what that's going to do is run is run task one and task two.

[00:13:52]Just like that.

[00:13:53]And you can see down here because these tasks don't error, it says the build succeeded.

[00:13:58]Uh the reason it says three tasks is because Invoke-Build sees my pipeline as its own task. So, we've got my pipeline, task one, task two. Results in three tasks, even though there's only two tasks actually doing work here.

[00:14:14]Excellent. So, now we're going to move on to an actual validation pipeline.

[00:14:19]So, when I was talking earlier about you can build a pipeline with 20 lines of PowerShell, this is this file here.

[00:14:25]So, for these demos, uh I've written a module called PS Service Desk. The module itself doesn't do anything. I just needed something to structure to run a pipeline against. So, just think of PS Service Desk as a proxy for any module that you work in, um and just sort of swap it in in your head. But, it's got some basic public functions, it's got a private function, and then it's got some config like a YAML file and a JSON file. But, again, it's purely just a proxy for you to imagine your own module in this scenario.

[00:14:56]So, the key thing here is our build file.

[00:14:58]So, we've got our PS Service Desk.Build.ps1.

[00:15:02]It's in the root of the module, so Invoke-Build will find it.

[00:15:05]And we'll just go through it now. So, at the top here, I've just saved um the PS script root into a script file for module root. You don't have to do this. My brain just that's how I like to do it. So, this is purely optional, but it makes it easier to explicitly call it the module root, so I remember where I am.

[00:15:23]And then down here we've got our Pester task.

[00:15:25]So, I in this case I've called it test.

[00:15:28]So, what that means is in my CLI I can just type invoke build test and it will run my entire Pester test suite. So, that again that just sticks in my head better than Pester.

[00:15:39]But, all this is doing in this example is looking through the test folder, pulling out every file, and then running through invoke Pester.

[00:15:47]And we use enable on enable exit. So, that means if Pester has an error, it will throw, and then invoke build will see that error, and then it will stop and go, "Hey, your pipeline's broken."

[00:16:00]And then down here we've got script analyzer. Again, I've called this analyze just to save some typing.

[00:16:06]And all this is doing is going through the module root recursively on every file and include the default rules. So, a bit of a hammer with script analyzer, but for this demo it's enough. In your real-world scenario, you'd probably want to dial in the settings a bit, but you don't have to. This works absolutely great. And I haven't used enable exit on invoke script analyzer because I wanted to save the output and just format it as a table. Again, just to demonstrate there's different ways of manipulating your output depending on the test you're doing.

[00:16:35]But, in this case I just wanted a table and then I write error, which is going to stop the pipeline and say, "Hey, there's something wrong."

[00:16:43]Then at the very bottom here we've got our validate task, and this is what I call the pipeline. It's the collection of all our other tasks that are doing all the work for us, and this enables us to run invoke build validate, and it runs our entire uh pipeline there.

[00:17:00]So, now we're going to jump back over to the CLI.

[00:17:12]So, what I'm going to do first is just run my test task. So, remember that was calling Pester?

[00:17:19]So, if I just wanted Pester to run, um, this abstraction is really great because I don't have to remember all the configuration that I need with Pester.

[00:17:27]It's all pre-configured, customized to the module exactly as it needs to be, and I can simply run Invoke-Build Test.

[00:17:34]It's a very, very useful when you're building out your tests, or again, you're in your flow state and you're trying to write something. It means you can hone in just on this one particular task, and make sure it's doing the right thing.

[00:17:46]Then, if I want to run the entire pipeline, I can do Invoke-Build Validate.

[00:17:54]And what this will do is run my test task, which is Pester, and then my analyze task, which is Script Analyzer.

[00:18:01]And because there's no errors here, you can see that it's just gone through analyze, hasn't done anything cuz nothing failed, and then at the end I've got my free tasks. So, that's my validate task, my test task, and my analyze task. And here you go. There's a validation pipeline in 20 lines of PowerShell, extremely simple to do.

[00:18:21]But the good news is we can get more complicated than that.

[00:18:27]So, we're going to jump back to VS Code.

[00:18:30]And in the examples here, what I've done is just copied and paste the PS Service Desk, um, module. So, again, you know, this representation of a PowerShell module just makes the demo a bit easier.

[00:18:41]But the key difference here is our build file has changed quite a lot.

[00:18:45]So, um, at the top here, we've got our variables.

[00:18:51]They're just convenience, really, just to make it easier. And then we've got something like code coverage. So, I mentioned code coverage, uh, so in this case, we just set it to 80, which is going to be 80%.

[00:19:01]Then we have a few helper functions. So, in this case, there's a very small function that's just working out the coverage percent. So, I think here, again, you don't need to focus too much on this exact code. It's on GitHub, you can review it. Uh what I really want to show you is how all the tasks uh tie in and um work together.

[00:19:21]So, down here we've got all of our tasks. You'll notice we've got test again. It's changed a bit this time and we're using proper Pester 5 syntax.

[00:19:28]That's because we want to use code coverage.

[00:19:31]But our analyze task for script analyzer, this one is exactly the same.

[00:19:36]So, if we look at this one, that's exactly how it was in the previous one.

[00:19:41]And then the original table that I showed you is now all of these tasks.

[00:19:46]So, for example, we've got validate JSON, which is part of our content group. And all this one is doing is finding every JSON file, loading it in, and doing convert from JSON. And then that stops me leaving a trailing comma and bringing down production. So, I really like this task.

[00:20:02]Um but the What you can also do is obviously extend this or add another task to do all your schema validation, depending on your structure and how you set that all up. Uh and you'll see in a moment how easy it is to expand and add on these tasks.

[00:20:16]And then we've got all our other tasks.

[00:20:18]So, this is our module conventions task saying the layout and the exports and the naming.

[00:20:22]The exports one is quite useful. Uh what it would do is check everything in the public folder, and then it will cross-reference that with the functions to export in the PSD1. So, when you're renaming files or you add a new function, it will make sure that you've declared them properly so they're exported.

[00:20:39]But again, uh we don't have time to go into the nitty-gritty of the code. What I really want to show you is these groupings.

[00:20:46]>> [clears throat and cough] >> So, these match the table I showed you earlier. So, we've got our code quality.

[00:20:51]So, that's our analyze, which is PSScriptAnalyzer. We've got test for Pester, and then we've got code coverage.

[00:20:58]Then we've got our release hygiene. So, this one's just checking your module version and your change log.

[00:21:03]Then we've got our JSON, again just validating that the JSON and YAML loads.

[00:21:09]And then we've got our module conventions, so just checking the layout, exports, and naming.

[00:21:13]And then I wrap all of these up again into another task called validate.

[00:21:18]So, these namings are somewhat arbitrary. It's just what I chose. You could call them anything you like. But, what I do like doing is having these subgroups cuz remembering the name of 10 tasks is quite difficult even after you're familiar with it. So, what you can do is just group them together and go Ah, yes. I want to test all my content. And then you can do invoke build content and it will test your JSON and your YAML. So, it gives you that extra level of flexibility, but you really don't have to do this. You could just declare them all in your validate task and you would just get all of the tasks all of the time. So, I purely do it just for convenience.

[00:21:55]So, let's jump back to the CLI and we'll see what it looks like when run.

[00:22:00]So, I'm in the same folder with the build file I just showed you.

[00:22:05]So, now we can do invoke build validate and it will run our entire pipeline.

[00:22:10]So, what we've got at the top here is our test and um analyze tasks, which we're familiar with now.

[00:22:17]And now it's added that all of the bunch of the other tasks. So, you can see this module has a very infusiastic enthusiastic 100% code coverage.

[00:22:25]But, all the others the reason that there's no errors is that it's run the task. There was nothing uh in the error stream, so it's all good and it's moved on. And you can see at the bottom here our build succeeded and we have 15 tasks. So, that's the validate task, the four grouping tasks, and then our 10 tasks which are actually doing something.

[00:22:47]So, let's move on now and show you a failure.

[00:22:51]So, again, I've now copied and pasted this module. So, this is exactly the same uh build file, so the full build file with the 15 tasks and the same module. The only difference here is that I've uh left a space in my YAML. So, this YAML is no longer valid and it won't pass. So, what we'll be able to see now is that error.

[00:23:13]So, we're going to move back to the CLI.

[00:23:15]I'm going to move into that folder with the broken YAML.

[00:23:19]And now we're going to run this pipeline. And what will happen here is on the YAML task, it will fail. So, you can see here that it's failed cuz it has odd indentation on line four, which is exactly what it has.

[00:23:32]So, now if we go back to VS Code, so I'm in the same folder with where the build file was, I can remove this mistaken space, save the file.

[00:23:44]I go back to my CLI, clear the screen.

[00:23:47]And what I can do here is actually just run the YAML task. So, when again, when I'm talking about flow state and you're in the middle of something, let's say you had 20 YAML files that you've just added and you've tweaked them all and now you're getting a validation failure.

[00:24:01]What you may not want to do is run the entire pipeline again cuz if your best to test take, you know, a minute to run, that's quite annoying. So, the beauty of breaking it all up like this, I can just run my YAML task. And because I've fixed it, it runs really, really quick and I know it works.

[00:24:17]So, then once I'm happy, I can move on and run the entire validate pipeline again.

[00:24:22]And now everything passes cuz I fixed the error.

[00:24:28]Great. So, let's move on to showing you how to add a task. So, as you can probably guess, I've copied and pasted the exact same thing, same build file.

[00:24:39]But in this scenario, uh we've decided in my team that every module needs to have a read me file. So, we want to make sure that that's validated in our validation pipeline because eventually someone will forget and will miss it.

[00:24:53]So, we simply go into our build file and we can add a build task.

[00:24:58]So, we just do add build task, give it a name.

[00:25:02]I'm going to call it readme. Sounds like a sensible name.

[00:25:05]And then we define jobs with our script block. In this case, all we're going to do is an if statement to see if that readme file exists.

[00:25:12]So, it's just standard PowerShell that we all know.

[00:25:15]And we can do a test path.

[00:25:20]And I'm actually going to use a join path as well. Um So, bear with me. Typing and speaking is interesting.

[00:25:30]>> [gasps and sighs] >> But, yes. So, I know I'm looking at the module root.

[00:25:34]And I know I want to look for a readme file.

[00:25:37]So, should be uppercase.

[00:25:41]Don't worry. I'll show you this again. I appreciate I'm moving a little bit fast.

[00:25:45]And then in my um if block, what I want to happen is it to error if that file doesn't exist. So, we're simply going to write error and say, "Hey, readme file not found."

[00:25:58]And then I'm going to get the script root so I know exactly what I'm referencing. Um Okay, brilliant. So, just to go over that, we defined the task, gave it a name. Then in the script block, we said the thing that we want to check. So, in this case, we're using test path to look for a readme file in the module root.

[00:26:18]And because we want it to run if it's not there, we invert the logic and then write error if it's not.

[00:26:25]So, then the only I need to do is add it to one of my groups.

[00:26:29]Uh I could add it directly to my validate task, but because I like to group stuff for the reasons I said, I'm going to put this under module conventions cuz I think this is probably quite an opinionated um opinion to have a readme. So, I'm just going to put that in there.

[00:26:45]And then that means now when I run invoke build validate because of the module conventions task here, this is going to pick up the read me task and run it in my pipeline.

[00:26:57]So, if we jump over to the CLI, we're going to go to the folder we were just in with that new build file I just edited. And now, I'm going to run my validate pipeline. And what will happen is it will fail because there's no read me file found in that module. So, it's as simple as that as It's as simple as that to be able to expand and build out all of these tasks.

[00:27:21]So, for complete completeness, I'm just going to come back here, add a read me file, and we can see it passing.

[00:27:34]So, there we go. We're now up to 16 tasks, and everything's passing. And if I wanted to, I could have just ran the individual uh read me task as well, which is quite useful when you're developing them to test them.

[00:27:47]But, because this is a small module, it's very fast. It's very easy to run validate as well.

[00:27:53]Good stuff.

[00:27:55]So, uh now we know how to run invoke build validate. I just want to show you how that applies in your CI system, and also with an LLM with an AI agent.

[00:28:06]So, first up, uh this is a YAML file describing um uh GitHub actions. So, this is configured to run when you push code. So, I um continuous integration pipeline, right? And as you can see, it's only doing two things. It's installing my dependencies, pester script analyzer, invoke build.

[00:28:26]And then, my entire validate pipeline is invoke build validate. And what that will do is it will run the exact same thing that I've just shown you, but in your um CI pipeline. And because it's, you know, you can have merge rules and all of that, it becomes the source of truth and that enforcement.

[00:28:44]So, this is a slightly different module uh called uh PS Pi Hole, but you can see here at the very top, it's run Invoke-Build-Validate.

[00:28:53]And this looks very, very similar to the output I just showed you. This is the simple one, which is running Pester and Script Analyzer. And then you can see at the bottom, build succeeded, three tasks, no errors. This will pass.

[00:29:06]And then here's an example of the full pipeline with our 15 tasks. And you can see again, it's run our tests at the top, it's run analyze, and then it's gone through all of our other checks.

[00:29:16]And at the bottom, we have our 15 tasks, all good. And you'll notice that I'm running this on both Windows and Ubuntu, just to show that it can be done, because all those three modules, Pester, Invoke-Build, and Script Analyzer, work on both.

[00:29:29]So, you get the ultimate amount of flexibility.

[00:29:33]And then, just to close the loop, this is here with a broken YAML file. So, this module doesn't have YAML, so I just put some random stuff in there just to show you, but you can see it failed in a very similar way. And then it stops the pipeline, and you can see at the top, the red X mean that this validate pipeline has failed in our CI.

[00:29:55]So, I don't expect you to read this, I'll talk you through it. But what this is is a skill that you can use in your AI coding harness, so something like Co-pilot, um Claude Code, or Code X. What you can do is create a skill to say that, describing to the LLM, that whenever you modify a PowerShell module, you must go into the module root and run Invoke-Build-Validate.

[00:30:20]And then what you'll see is that iterative behavior, so if the LLM decides to do something it shouldn't do, the deterministic validation pipeline is going to force it to acknowledge the error and fix it. One interesting thing I noticed is for a older module when I run it and had quite a few script analyzer errors, because it was like legacy code, rather than fix it, the LLM was like, "Oh, I'll modify build.ps1. Now it's passing."

[00:30:47]So, the way to mitigate that is you can directly say, "Do not edit build.ps1."

[00:30:52]And it, from my experience, it generally respects that. Um so, you just have to really spell it out because LLMs can be a little bit tricky and try and work around you.

[00:31:02]But, um I've been using this skill and it works fantastic. Automatically picks it up, automatically knows to run it, and runs my validate pipeline exactly as I've just shown you in those other scenarios.

[00:31:14]And here, this is a live example of this running in Codex, again on my uh PSP home module.

[00:31:21]So, what it did first, uh the new PSP home domain body, it called it something like create-pspi home body or an unapproved verb. But, because of that skill, it then ran invoke build validate. Uh script analyzer was like, "Hey dude, what you doing?" So, then it fixed itself. Um and then it ran invoke build validate again. And you can see that it's running this here.

[00:31:45]So, it included the file, but you can add the file, but you don't need it. We can use the shortcut just by using invoke build um the context it's in.

[00:31:54]But, in this case, I showed you it's running invoke build validate. And then what happened here is the module version check failed because I hadn't bumped to the next version of the module. So, you can see here it's then asking me to make this change. And again, this is a really, really useful feature. Because we brought our validation pipeline out of our CI and we've unified it in a single place, it means tools like LLMs can actually run it and then sort of fix themselves and iterate and build on that.

[00:32:23]And it's extremely useful.

[00:32:26]Brilliant. So, the next steps. I think your key takeaway, if you remember nothing else from this talk, would be to make your validation pipeline a single local command. Hopefully, I've convinced you of the advantages of doing that and make it worth the time. And if you don't have a CI or a validation pipeline now, I would strongly encourage you to create one local command invoke build validate.

[00:32:48]As you saw, 20 lines of PowerShell will get you really far.

[00:32:52]And then um oh yeah, Christian tomorrow has a talk on script analyzer. Just wanted to call that out if you wanted to learn more and if not you script analyzer before.

[00:33:01]And then we also once you're running those two, you get a feel for it, you're comfortable, you can then expand it out uh and do your JSON version and change log and all the other tasks that I've shown you next.

[00:33:13]Then if you already have a CI, um I challenge you to move that validation logic into invoke build validate. You can make your CI call the same command that developers and LMs can run locally.

[00:33:26]And all the code that I just went through now is available on GitHub. It's on So if you want some inspiration of the tasks you can use or where to get started, you should be able to just copy and paste that build file into your module root and start using it straight away.

[00:33:45]Brilliant. I think we've got some time for questions if there are any.

[00:33:49]Yes.

[00:33:59]Yep. So the question was does invoke build support warnings?

[00:34:03]Um not directly as sort of a mechanism.

[00:34:06]It's either pass or fail, yes or no. But two options you could use was literally use write warning and then it will write out in yellow text if something's not right. Or another option, a more advanced feature in invoke build is if you use a question mark in the task name, that means continue on error. So it would actually run the task um um and then you you get all the output at the end and then you can collect it. So, I'd probably say the simplest thing is just leverage right warning, and then in your logic in your task, you write that in, and you'll get your warnings as you need.

[00:34:42]Yes.

[00:34:46]Yes, the question was how do we debug a custom task? So, yeah, really good question. So, what I normally do, um and I left it out of here for simplicity, is actually move my task into uh functions or separate scripts, um because when you run Invoke-Build, it actually runs in a separate context. So, troubleshooting it sometimes can be a little bit tricky uh if you're not familiar with its internals. So, my recommendation would be to move the logic into its own script, then test that, make sure it works, and then just call that from the task.

[00:35:21]But, a lot of these tasks were really simple, right? So, um they're pretty pretty intuitive if it errors. But, yes, as you build out your tasks and they get more complicated, that's what I would do is separate it out.

[00:35:35]Yes.

[00:35:43]Yes, so the question is uh how do you share your tasks across your repositories? Cuz you probably want to reuse them and not write them um again and again and again. So, for simplicity sake, I just showed you a build file, but the next evolution of this would be to write a wrapper around Invoke-Build, and that contains your tasks. So, you'd build a module uh that contains all of your tasks, and then the build file in the module would reference that module. So, that way you have all your tasks in a safe central location, then you just dot source them in, and it works exactly the same way.

[00:36:15]So, um I didn't really have time to show that in detail, but that's how I would abstract it. You just move them into their own module or NuGet feed or anywhere else, and then just dot source them into the build local in the module that you're working on.

[00:36:34]Excellent. Well, thank you for your time.

[00:36:36]If you'd like to leave me feedback, you may. I'd appreciate it.

[00:36:40]And yeah, thank you for your time.

[00:36:41]Cheers.

[00:36:49]>> [music]