Rubber Duck Thursday: GPT 5.6 models in GitHub Copilot

[00:05:14]Hello.

[00:05:16]Hi. Hi everyone. Good morning. Good afternoon, good evening depending on where you are joining us from. Welcome to this Thursday's first instance of rubber duck Thursdays. I say first because we stream this live stream across three time zones. So this is the first today and yeah it's great to connect with all of you. Thank you to everyone who's joined. I see we have so many people joining and already active in the chat. This is great. So welcome, welcome, welcome. Um, this is our weekly live stream where we get together, talk all things GitHub, talk all things GitHub copilot. If you have any questions, this is the place to pop them in. And if you have like any demo requests, this is also the right space for you all. So that's fantastic. I see you have so many people that are joining from different parts of the world. Uh, we streaming this on YouTube, Twitch, and LinkedIn. So you can join in on your preferred platform of choice and the live stream will be accessible after we are we are done um through the same link. So I see we have some users joining in from India. Chennai welcome to the stream. We have a user from Australia. Good morning Muhammad. I'm seeing some familiar names. So welcome back.

[00:06:39]All right. Um Ali just crashed. So session crashed here. Welcome to the stream. I hope you'll find it useful. Uh if you have any questions, uh also feel free to start popping them in the chat and we'll continue addressing them as we proceed. Have users from France.

[00:06:57]Uh Chenal is asking when we'll start. We are live. We are good to go.

[00:07:03]Fantastic.

[00:07:07]All right.

[00:07:09]All right. So you say it's a good evening. Well, depending on where everyone is joining from. For me, it's actually just good afternoon.

[00:07:16]Okay, so I think we'll get started. I'm going to pop my screen right here. Uh, so first order of business is we're just going to go through the change log. Just briefly discuss what's new, what's exciting in the world of GitHub. And I'll just keep monitoring the chat in case any questions come through. So we'll ch we'll just start here from the change log. I'm going to drop this link for you all on the chat in case you want to follow through or just to read more about some of the things that we discuss. Now I was going through this page earlier in preparation for this live stream and I noticed something that I think is probably interesting. uh you'll see that this week we have so many new releases, improvements, announcements and most of them are actually focused on security. All right, so that just goes ahead to show how critical security is at the moment. So yes, it's not all about the excitement using coding agents across the full SDLC, but we're also trying to ensure that our security products are at par and that we're not opening up any attack surfaces for coding related attacks. So that's just an observation I made from the change log this week that most of the announcements and improvements are actually security focused. So let's just talk about some of these announcements.

[00:08:43]We won't go through everything uh everything on the page, but you have the link. You can just revisit that um whenever you'd like to.

[00:08:52]So, the first one here is just some improvements to secret scanning and public monitoring. Um something we can look at probably. Let's see if that's you can talk a little bit about that.

[00:09:08]Uh yes, it's it's just some secret scanning improvements here. So yeah, probably that's something you can read more on. Um, GitHub copilot in Visual Studio. This is not security related, but I think for the Visual Studio users, I myself use Visual Studio Code and the new GitHub Copilot app a lot. But for those on the stream who use Visual Studio, then June's update had quite a lot to offer. So it had a lot in terms of improved usage tracking and alerts.

[00:09:41]So you now get like an alert as you're as you're approaching your limits. This you just don't get your sessions cut off. Um there's a new trust validation layer for your MCP servers. So this way if anything changes in one of the trusted MCP servers that you've installed, then it's going to prompt you to review the new changes. So this goes a long way in it's actually like a security layer. If anything changes in the underlying MCP servers that you have installed in your environment, you will have visibility on that. Uh then there's this interesting one around the GitHub copilot modernization agent for C++. So it's now in GA. So [clears throat] if you're on Visual Studio, you can now use the GitHub copilot spec this is a model that is specified and optimized for modernization tasks. And I'll go ahead and admit that we do have a lot of legacy code bases out there. So this is a good opportunity to test and try out a mo or or an agent that is optimized for just modernizing legacy code bases and um upgrading them. So that's an interesting bit. So there's quite a lot as I said for the Visual Studio users of Copilot. Again the full change log is available on this page. You can read more about what that means.

[00:11:08]All right. Um, let's see.

[00:11:11]Um, code scanning now shows AI security detections on pull requests. Let's see what that's about.

[00:11:23]Okay. All right. So, now this one is interesting. Um, so code scanning now shows AI security detections. Why is this important? I think just to build some bit of context um we have some built in or we have some supported languages and frameworks that currently work with CodeQL. So CodeQL out of the box doesn't work for all languages for all frameworks for all libraries. So we have this list here of the supported languages and compilers. So you can see it's a fairly short language uh language list compared to the full list of programming languages that we all work with as well as the frameworks for each language. So this list is limited. So with this new is it a release? It's a new release. With this new release, this means that we are extending CodeQL to now use AI to basically detect any vulnerabilities to languages and frameworks that are outside this list.

[00:12:26]Okay, so I think that's pretty useful.

[00:12:29]Uh that means that when you're using um copilot code scanning, then it's going to surface AI powered security detections directly on a pull request.

[00:12:39]And this is as an expansion to the list that is currently supported. So if you're worried that your language is not in this table, then with this new release, CodeQL will actually use AI to do some extended um some extended checks on these languages that are not in that primary list. So that's something worth reading. And yeah, if you're using if you're using code, if you're actually a user of GitHub advanced security, that is a feature that you might want to read about. But yeah, that's that's pretty interesting.

[00:13:18]Um, probably the last one that we will look at cuz I know that we have quite a lot is dependable version updates now introduce default package cooldown. Now let me just read about this a little bit because it's quite interesting. So currently dependabot will open up here to update your packages as soon as these packages are released on the registry.

[00:13:44]But what we know is that new releases are a common entry point for supply chain attacks where compromised or broken versions can reach your dependency updates before maintainers and the community have caught it. So that window when a new version hits the registry is where the attackers probably target to launch um to launch um attacks. And so what this improvement improvement does is that currently dependabot will now wait for at least 3 days. So after a new version hits the registry, dependabot won't open up request immediately. it's going to wait at least 3 days before it opens that pull request. So that means within that 3day window, most likely the maintainers or the community will have caught any critical vulnerabilities that are introduced in that new release. So probably that's something that's I think very useful at at the moment. So that's probably just a good improvement to note.

[00:14:50]Um, yeah. So, I think that's it about the change log. Of course, there's so much we haven't even covered, but definitely take the time to read through the change log and see what's new.

[00:15:03]So, today for this live stream, I'm just looking through the questions on the chats.

[00:15:16]Um, June updates [clears throat] All right. Yeah. So is just summarizing the June updates there on Visual Studio.

[00:15:25]So there's there's quite a lot as you've seen from from the announcement. So that's pretty interesting if you're a copilot user on Visual Studio and show you on the latest version and try out this new um features that have been rolled out.

[00:15:39]Just scanning through.

[00:15:42]All right. And in case I'm seeing comments coming in from the three platforms you're streaming to. In case I miss your question, just feel free to repost it on the chat. Okay. So, probably as you saw from the title of today's live stream, um last week GPT Open Air announced their new model family that is GPT 5.6. And so, we're just going to spend some time and experiment with these new models. Um I don't know how many of us have actually had a chance to try the new GBT models.

[00:16:14]So, let me know on the chat if you've had a chance to try either GPT56 um either Terra, Luna, or Soul. If you've had a chance to test them, let us know what is what what you've learned from the models, what works, what are their strong points, and then we're just going to tinker around with these models on the live stream.

[00:16:37]So, the surface that I'm going to use is the GitHub Copilot app. Now, if you've not used the GitHub copilot app, probably what you want to try out is just visit the link gh.io/up and this should uh let me just drop that on the chat.

[00:16:57]gh.io/app.

[00:17:02]And you should be able to install the new copilot app from here. And so that's what we're going to use to play around with these new models. So from the app, as you can see, the interface is really designed to allow multi you to allow the management of different agent sessions.

[00:17:23]So the idea here is that it's agent native. So if you're having multiple agents working on different work streams for the same project, then you have this consolidated view of all the different um sessions that are running for each of your agents. And so we're going to see that practically as we proceed. So the demo that I had prepared here and uh I actually saw this would be the best uh chance to to test this models out is there is new this project I've been working on. I'm going to launch Visual Studio Code to show you.

[00:18:02]Just going to bring this over.

[00:18:07]So, what I'll do, I'll just create a new file or let me just open a project.

[00:18:12][clears throat] So, I want to show you the project that I've been working on here.

[00:18:17]Labs.

[00:18:20]Let's [clears throat] do Uh, okay. Let's let's just do a simple project [snorts] and I'm going to create a new env file.

[00:18:39]Okay, I'm going to zoom in a little bit just so this is visible to everyone.

[00:18:44]Then let me just add some details. Let's say I want to save my secret.

[00:18:51]my secret.

[00:18:54]Um, that's my secret. I have a key.

[00:18:59]So, that's my key.

[00:19:02]Now, I do a lot of publicly facing demos. And thing is, of course, you don't want to open up your TNB files when you're in the middle of a presentation such as this one, but sometimes you can accidentally just click on one of those files and then end up exposing your secrets and your keys.

[00:19:21]So, what I'm working on is just an extension that I get on VS Code. At the bottom here, you see this toggle demo mode um option. And if I click on that, then it automatically just masks anything that appears to be confidential. So if I'm doing my demo, I can go about peacefully just navigating the different files. If I'm trying to explain uh let's say it's a deployment on Azure, which are the secrets, the credentials you should be working with, I can safely just expose myv file without the risk of accidentally um showing some of my keys. So that's the extension and then on top of this so this is like for the editor on top of this is right now most of the work happens in the terminal. So if I launch the terminal and right now if I do a command like a account list. So if you use the Azure CLI if I run this um if I run this command it's actually going to print out everything regarding my different Azure subscriptions. So, it's going to print print out the subscription ID, the tenant ID, my email address, everything will be shown here. But with this extension, I have um I have this private terminal instance where I can pretty much do the same thing, but then on the terminal, the extension is going to mask again this sensitive information. But as you see here, it doesn't seem to work. like I just discovered there's a bug in this um particular feature for the terminal. So this is what we're going to test out with one of the new models from Open EI and see how well it's going to handle this task. So I'm going to switch over to copilot app. This is where I'm going to start my work from. So I have the project here. So the project is called key mask. It's an extension. It's not yet published. So the code and everything is currently just running locally but it's in this folder key mask. As you can see I don't have any active agent sessions. So I can hover on the folder name and click on this plus sign. So that will start a new session.

[00:21:40]It's going to start a new session in a new work tree. So it's going to branch off domain and then it's going to assign an agent its own isolated space to work.

[00:21:50]so I can easily experiment around. So I'm going to just send like a simple prompt.

[00:21:58]I will ask it to summarize this project.

[00:22:05]So I'm doing that so that uh you see that the work tree has been created.

[00:22:09]That's the first thing. Number two, at the very top, I don't know if you can see this, but at the very top, uh there's the um session metadata. So, this is where you can see information about this new agent session that you have just started. So, we have the session name. Um I can see the token distribution. So, these are the input tokens, the output tokens. I have my context um distribution. So you can see that the system prompt here takes 2% of the context window. We have the distribution across the system tools, MCP tools, messages, and everything else. So if you've used the app and you've probably not used this view before, it actually gives you a pretty good breakdown of your context window and how the tokens are distributed. So you can easily just manage your um your your token usage from this view. This is not the only available view. Again, right here, you can open this panel. So at the very top right, there's this button that allows you to open the panel. And then one of the available views that you have is this insights tab. Okay, so we have this insights tab and it gives you a pretty much um similar view of your context window. So you can see as the agent works, you will see the different um consumption or yeah, you'll just see how my context window is going to fill up, you can see how the different tools are actually being invoked by the agent. So it's just one view that will give us good insights in terms of what's happening as the agent works. So I'm going to run /cle so that I can clear my context window and start from a clean slate.

[00:24:06]So we'll do that. We'll clear the session. And then I'm going to now use one of the models. Uh and let's try this with terror. So Terra is the model that is in between um soul which is a powerful model quite expensive and then oh I hear you can't hear me.

[00:24:29]Uh can you confirm that you're not getting my audio and thank you for raising that up.

[00:24:41]So, can you just confirm? Maybe let's get one more person to confirm if you can hear me.

[00:24:50]I see Cher is saying she can't hear me.

[00:24:53]So, probably let me know if you're not able to hear me.

[00:25:02]Um, hi Kishka. Hi everyone. So, just confirm if you can hear me, if my audio is coming through.

[00:25:14]Okay, fantastic. Thank you. Thank you.

[00:25:16]Sorry. Um so probably um Cherry, you could confirm if one of your um audio um output devices is failing. Okay, great.

[00:25:26]Um yeah, so I was saying I'm just going to test out the terra model. So in this case, for those who have just joined, um we are testing out there's this local extension that I work with. Ideally, if I switch on demo mode on VS Code, it should just automatic automatically mask my different credentials and and keys.

[00:25:48]So, this way I cannot accidentally just expose some of my information at a public forum such as this one. So, there's a bug here. So, you can see it's supposed to work even in the terminal, but it's currently not working. And uh let's see. See, it can't even allow me to type. So, I'm trying to type, but it's it's misbehaving. So, let's try and see how the model is going to handle that task.

[00:26:19]Okay. Um, so that's it. We are on we're on a new context window. So, this is a new chart.

[00:26:27]There's nothing yet. So, we'll switch over to GPT56 terror. For the reasoning effort, I will leave it to high. Then I'm going to provide just a prompt describing the issue that I am facing.

[00:26:41]So the private terminal, private terminal is the one that has an issue. Everything else works fine. So the private terminal um is busted.

[00:26:55]It doesn't correctly pick up the the path name uh the path and I am not able to type in I'm not able to type in my command. Okay.

[00:27:21]Um, so investigate.

[00:27:24]Investigate and fix this.

[00:27:28]For some reason, I can't type. So, let's just send that in. And as soon as we send our prompt on the right where we have the insights tab, we should be able to see the activity, the agent activity as it's being streamed in. So, we can now start seeing our context window start to fill up. or the very extreme right you will see these blue dots and a map that's forming. So I can easily now just monitor my context window. If I get to a certain percentage I can think about whether I need to compact that. Um so that's just easily um accessible from this view. So we'll see that the agent is loading some skills. So it has loaded the impeccable skill. I'm not pretty sure why it's loaded it. Um, so let's see. There's a question here. So for Kimas primary product surface, should I capture?

[00:28:25]Huh?

[00:28:32]I'm not quite sure what this question is about. Uh, should I capture this?

[00:28:39]It doesn't seem relevant to my prompt.

[00:28:45]Okay. So let's see the terminals back end is custom pty code. So I'm tracing its profile provider launch configuration. All right. So it's it has now picked up the issue. Not pretty sure. I'm not sure what what that um question earlier was about, but let's see.

[00:29:06]So the agent is now working on the right here. We should be able to see how it's consuming the context window. And I think I saw a comment on context window on the chat earlier. Let me just scroll up.

[00:29:33]Okay. Okay. So, Alex, I've just seen your comment here. Uh, so why does Microsoft have to brand everything as copilot? It's making it really difficult to work for users to request access.

[00:29:43]Yeah, I I do admit that we probably have over 70 products uh right now with a copilot name slapped all over them. So, that I do admit is something that can probably be be fixed. But at the moment, unfortunately, it's it's not like um probably that's the marketing team that will we'll have to see how they can they can uh rework that strategy. But for now, just have to be very specific. For instance, we're talking about GitHub copilot. So, not M365, not Microsoft Copilot. The scope for this session is GitHub Copilot. So the best that we can do at the moment is just um prefix copilot with the specific product that you're working on. But yeah, that's a valid concern. I'm also hoping it will change. Um there is a question here. Is um Mi code one flash from Microsoft? Yes, that's one of the um models from Microsoft research. So you can definitely give it a try if you haven't um and and see how it basically performs. So code one flash is specifically um optimized for coding um workloads and it's actually optimized for like VS code and this particular tooling. So you should see some bit of difference when you're using um the the Microsoft research models for your workloads.

[00:31:09]Um so let's see uh the first thing that I observed about the GPT56 models is that they're fairly fast. So across the different experiments I've run with them, they are really really fast. So I don't know if anyone else has made that observation, but let's see. So uh the agent was able to find the the input loss and it claims to have fixed the private terminal. So let's let's start it.

[00:31:40]I'm going to open this instance. So again I said earlier on the co on the GitHub copilot app all these sess sessions are running in isolation. So if I want to give another agent another task to work on the same project it's not going to work on the same version of the code as this agent is. So this agent is working in its own work tree. So I'm going to open that work tree in Visual Studio Code Insiders.

[00:32:10]So this way it's going to launch uh using a different window. So this is the work tree. This is the environment that this specific agent session is using. So I'm going to go over to the debug and run the extension. So it claims to have fixed the terminal extension.

[00:32:30]So let's see. Let's see if it works.

[00:32:35]So this again opened a different windows. too many windows are popping up. So that has opened a new window and from here demo de demo mode is toggled open. So if I open av file um I expect for it to automatically mask the keys. So that functionality is still available. Now let me open the terminal and then I'm going to open a private terminal.

[00:33:06]Um, so let's see.

[00:33:09]Yeah.

[00:33:13]Yep. It doesn't work. Doesn't seem to work. So I'm not sure. Let's Let's restart it again.

[00:33:22]So run the extension.

[00:33:26]Give it another try.

[00:33:31]Um, so open private terminal.

[00:33:37]Okay, now I can't open the private terminal.

[00:33:41]Private terminal.

[00:33:45]And yeah, so we can see Terra was not able to fix that issue. [clears throat] Let's see. Let's see if we can try and steer it in the right direction or let's tell it. The issue doesn't appear to be fixed.

[00:34:10]Okay. New.

[00:34:13]Let me just leave it at that and see what it does.

[00:34:20]Okay. I'll just go I'm just going to look at the chat again.

[00:34:30]Okay.

[00:34:38]There's a question from Farida. So, where can I learn about GitHub from scratch? I will point you to there is a YouTube um there's a YouTube playlist that has a GitHub introduction series.

[00:34:56]So, let me just pull that up for you as we wait for the agent here to uh look at what it might have missed from that first run. If it doesn't catch it, we're just going to discard this and then we can test it out with a powerful model that's GPT56 soul and see if it's able to do a better job.

[00:35:17]Um, so there's the if you go over to the YouTube channel, uh, to the GitHub channel, sorry.

[00:35:34]Um, so we should have a GitHub for beginners and I'm just going to pull this over to the chat.

[00:35:45]Yeah. So Farida uh this is the link that you could probably use. It should give you a very detailed and comprehensive um introduction to GitHub as a platform and everything you can do from here. And if you just go over to the GitHub um channel, we do have several playlists.

[00:36:04]So if you want to learn about the copilot CLA, you want to learn about um GitHub copilot for beginners, you will definitely find like the right series or playlist for your needs. So um Farid, I hope you get a chance to check that out.

[00:36:24]All right, so Baki is asking a good question here. Can you can you increase the verbosity of code changes? It looks like you can't approve changes or see them. [clears throat] All right. So, I'm assuming this is from this view from the copilot app. So, let me just expand this panel session. So, we do have the changes tab. And from there, you should be able to see all the files that the agent touches. So, if there's any change, you should of course see the the usual div view that we are probably used to seeing. So you can have access to the different files that the agent is working on. And then you can of course use the files tab as well just to see the full structure of the code. So it's not completely um we're not taking that away. You should be able to have a way to interact with the code files. You can see the changes as they're stream streaming through and as the agent works. It's just that the interface is not like the typical VS code interface where by default it's designed to support an like a human developer, let me put it that way, a human developer who's actively writing code. Here the primary user for the copilot app are your agents and as they work you have this visibility on the files that they're attaching the changes they're making and as I said we have this concept of work trees where right now if I were to start a different task use a different agent I'll just start a new session which will branch off from main and create its own work tree for that agent so they're not going to step on each other as they're working on the same code. So I hope that Back gives some bit of clarification in terms of um how much access you do have on the actual files.

[00:38:18]Um so let's see. So it's asking me to reload the extension host. Okay. So now let's let's try it again.

[00:38:30]Okay. So before I do that, I see some questions coming in. U Magnus says it's an interesting model. I think so too, but let's first see if it's going to be able to um fix this. In terms of the context window, it's it's used about 31% of my context window. So, let's see if it's actually managed to fix the issue.

[00:38:59]So, I'm going to launch VS Code. I hate that it's launching very many um VS Code instances, but it's fine. I'm going to stop the extension development server.

[00:39:12]I'm going to run it again.

[00:39:15]It of course pops up a new window. And there I now see some Okay, that's okay.

[00:39:22]Not yet. So, let's see if we can launch the private terminal.

[00:39:30]Okay, so the first issue appears to be fixed. Um, so before it wasn't able to correctly pick and display the correct path. So that appears to have been fixed. The second issue was that I wasn't able to type in my commands on the terminal. So again just for context for those who are just joining the live stream um we are using testing out the new GPT56 models. We started with GPT 5.6 terra that's like an intermediate level model. It's not expensive as 55 level 56 soul level but it's also not lightweight. So it's somewhere in the middle. So now we're going to test if I can um successfully type in commands.

[00:40:18]And looks like I can do that. And if I hit enter, the expectation here is that the extension is going to mask the any sensitive information that should be displayed on the terminal. So let's do that. And there so I see that um my tenant ID is now being replaced by this um by this text. Same for the ID. Uh same for the tenant ID. My email is now being not being displayed because otherwise running this on a normal terminal instance, this would have just displayed all this information for everyone to see. So looks like Terra was able to find the issue and fix it. So probably the last thing I'm going to ask it to do is right now I'd want for this text to be a bit more descriptive. So I want to know what it is that has been masked or what value should have ended up there. So let's switch back to the app and I'm just going to follow that with a prompt.

[00:41:20]Um let's replace let's replace this placeholder with a more dynamic descriptive name.

[00:41:34]I'll I'll just send that in and then at the same time at the same time I'm going to start okay so there's a question that has just been popped what should I what should a mask G ID display for example a generated label so I'll just tell it to um infer to the key to the name of the key example tenant ID.

[00:42:11]All right. So at the same time I'm going to start a new session. Um so now I want to create like a website for this for for this um this extension. And one of the things that I want to test out from OpenAI's announcements around this new model family, there's a claim that these models are now better in terms of design work. So before, I don't know if you all had tried to use um any GPT models for your UIUX work, they weren't really doing a good job. So from the announcement blog from Open AI, this new family um appears to do a better job at handling UI work. So let's let's give it a test. And in this case, we're now going to start with their smallest model. So we're now going to test this with the Luna model. So let's do that again from the Copilot app. In my folder, I'm just going to kick off a new session. So this is a new session that will be launched in a new work tree. So at the bottom here I can say yes launch this in a new work tree. I'm going to switch over to uh under lightweight I should see GPT56 Luna model. So it's more affordable.

[00:43:31]It's smaller. It's more lightweight. So let's see how good it will actually handle how it's going to handle this task. Then I'm going to switch over to plan mode first. Let's begin by generating a plan.

[00:43:46]And I'm going to instead of just using the agent as is, I'm actually going to use there's a built-in uh skill called impeccable. If you've not used this, I think it's something worth checking. So, let's impeccable style. And I'll just paste this on the chat.

[00:44:11]So, impeccable style. The idea here is that this agents can use this skill to move away from this um UI slope that you get from agents. You'll probably know that if you're using an agent for your front end work, it will have a dash of purple at the moment. So, we have like so many patterns that um AI models basically use for your for their UI and UX work. So with impeccable, it's actually um forcing the agent to move away from these common patterns and have some better outcomes. So I'm going to give Luna here um access to that impeccable skill. The good thing is that from the co-pilot app, it's actually a builtin skill. So if you go over to your settings, look at the skills under built in, we do have the impeccable skill that is that comes out of the box with the application once you install it. So no configuration is required. So in my new session here, I'm just going to use the slash command and I'm going to use the slash command here and look for impeccable.

[00:45:21]You can do a bunch of activities with your skill. You can have the agent here um audit the the quality of your UIs. It can do some last minute minute polish.

[00:45:34]It can distill. It can provide a critique on your UX. So in this case, I'm actually going to ask it to craft.

[00:45:44]So with craft, it should shape and design like the entire feature end to end. So that's what I want it to do. So, I want it to I'm going to just tell it that I need a simple website for this project to be hosted on GitHub pages and use remote for any marketing marketing assets.

[00:46:15]Okay, so now I'm using the smallest model from that family that's GPT 56 Luna. I'm in plan mode. I'm going to launch this in a new work tree. So, I'll just send that in. And so, you'll see now I have two different agents working on the same project, but they all have their work streams. They won't at no point will I have to manage any conflicts between the two agents that are working. So, again, I'm going to open the insights tab. I love to just have this side by side so that I can manage or just have a better view of how my context window is filling up.

[00:46:54]So, let's see how Terra is performing here. Let me see if we have any questions. Oh, there's quite a lot on the chat. Um, what's the differences from the previous model?

[00:47:13]I assume that you are referring to this new family of GPT56 compared to 55. So one is in terms of performance they are basically better from my experience. They are faster. So you'll get um you'll get faster responses. It's just a fast experience working with these new models. And then there's also um the price point. they're more competitive in terms of the um per token input and output cost. So there are some differences um between the different families. There is a question for us here from workstream be which is around designing the the website. So so should the GitHub pages site primarily serve? What should the visitors do first? Um so yeah it's just for developers who want to install the extension that's our target audience.

[00:48:07]What brand personality should the site project project uh should should the site project? Now, as you can see here, the agent is not just jumping into doing some UI work through this impeccable skill. It's actually now guiding and steering the agent here to think more critically about how to position or design this experience. So, brand personality, um let's go with playful, colorful, and approachable. See what it comes up with.

[00:48:37]Um any other question? Uh which visual lane should we explicitly avoid so the page feels distinct? Okay. So generic dark cyber security dashboards with neon green. Yeah that feels like a generic pattern for such a project. So I will definitely tell it to avoid that.

[00:48:56]Um accessibility baseline which one should the uh should the site use?

[00:49:03]Uh yeah, let's let's just use the first one, the recommended option. And in real time, you can see the tools that are being invoked on the right. What should the standalone GitHub pages site be built on? Um yeah, it recommends Astro, so why not?

[00:49:24]Okay, let's see. How should the remote marketing assets appear in the site? um your call.

[00:49:37]I don't care so much about that. So, I'll just have the agent make a decision there just for illustration purposes.

[00:49:47]Okay, I think that's all.

[00:49:50]Let's see how the agent let's see the plan that the agent comes up with.

[00:49:57]All right, so it's now working on a plan. So, let's just give it some time, see the first version of the plan, and then just send it off to work. Right.

[00:50:06]So, the plan is ready. I can view the full plan. So, I'll just click on that.

[00:50:11]And I love this because at the same time before I give the agent the green light to work, uh it gives me this plan view.

[00:50:19]So, I can see the full plan. So, the details, uh this is the problem that it has identified. This is the approach that it's going to take. It will create a standalone project. Um, this is the theme. So, probably it's inferring this from the interview just conducted with me the user. Um, yeah. So again, the the ideal scenario is that you'll spend some bit of uh some some more time on the plan just reviewing it and ensure that it accurately reflects what you want the agent to build. So, what I'll do is I'm just going to send it off to build it with autopilot. So, this way it won't have come back to me with so many questions. It's just going to have like um access to just run everything it needs. So, I'm just going to approve the plan and send it off to work in autopilot. And as the agent works, there's this view over here where you'll see every milestone in that plan. as the agent works, it's just going to check it over and I should have like a full uh green check here as soon as the project is done. So, I'm just going to give it one last instruction. Launch the website when done.

[00:51:41]And then I'll go off and do something else. So, the agent here has everything it needs to complete that task. So you can already see the experience on the copilot app. It's designed for you to be able to manage multiple agents running in parallel across your different projects.

[00:51:58]There's a question here. Is it a free version or paid? So you would need to have a GitHub copilot subscription for sure. But actually scratch that scratch that it is now available on across all plans. like if you're on Copilot free, she'll be able to use the copilot um the copilot app. But if you don't have a GitHub subscription, we do have the uh the functionality for bring your own key. So if I go over to the settings, there is the model providers tab. So if I click on that, you see that you can come in and add your provider of choice.

[00:52:39]So assume you don't have um GitHub hosted models through your GitHub subscription that means you can come in here if you have models say hosted on anthropic azure penai you have local models running on foundry local then you can set up your models and then use them use them with a copilot app. So, that's a good question. Um, probably something I should have mentioned earlier on, but yes, if you don't have a GitHub subscription, we do have the elements of adding and bringing in your own key to connect to your model providers.

[00:53:19]Okay, there's a question here. So, how do we do testing and roll back if we didn't see the expected result in it?

[00:53:27]Can we also maintain multiple versions?

[00:53:30]That's a good question.

[00:53:33]So now assume assume that whatever the agents did here so in its fix for this um for this extension we're not happy we're not happy with the work. So you can easily just um start a new session and that will create a brand new work tree. So the idea with work trees is that if it gets to a point and you feel like you're off in the wrong direction, you can just discard that work tree, you won't lose your code base and then just have a brand new work tree set up for your project. Now of course the overhead there is that you'll end up um using so much of your memory, right? So uh and and sure like a good practice is as soon as you're done with your work trees just delete them to free up that space from disk. But yes, the idea is that you can have your agents run wild. You can do a lot of experimentation with the agents.

[00:54:30]But if something is not going as desired, if right now I want to revert these changes, right? So I don't know if there's like a button to do that. But yes, I wonder what reset does because if you do clear, it's just going to clean up the [clears throat] context.

[00:54:53]the context window. It won't necessarily undo the changes, but nothing here is committed. So you like you can treat this as a safe environment to experiment. If it doesn't work, throw that out, try a different work tree. And it's basically um it's basically just going to to meet that purpose in terms of just rolling back and testing out different approaches.

[00:55:21]Um in terms of maintaining multiple versions, you can commit. So just commit as if you have a state where you're happy with it, just commit. You can ask the agent here to commit and then you can manage your different commit histories from from from the same platform.

[00:55:41]Um, so let's check on our Luna model how it's doing on the website plan.

[00:55:52]So it's still it's still cooking. Let's look at the insights.

[00:55:56]O, it's more than 50% of the context window used up. Okay, so there's something that I wanted to talk briefly about uh before we wrap.

[00:56:10]Right. So, okay. So, now our our terminal, our [clears throat] bug has been fixed. Uh, I'm going to invoke a sub agent to do a review of what Terra did here. So, from the change log, we didn't get to talk about this new announcement, but we do have the security review builtin agent that is now available on the app.

[00:56:35]So, I'm sure you can see right now that Chrome has just popped up on the screen.

[00:56:39]Uh, this is Luna just trying to test out and um experiment with the website. I'm just going to kick that out of the way and let Luna do its thing. So, security review. So I can use this slash command and this will just send off a a sub agent to do a security review on the current changes that the agent has implemented. So we'll hit enter. I'll switch over to the insights tab.

[00:57:12]And what you'll realize here is that we have just kicked off uh security review.

[00:57:18]Now this is not going to um be part of my main context. uh or my main agents context window. So what do I mean by that? This security review is being done by a builtin custom agent. So my main agent, the one that I've been working with, this is the context window for that agent. Now after invoking the security uh the security review agent, uh let's just open this and you'll see that this is the prompt that my main agent sent to the security review agent.

[00:57:49]I want you to perform a security focused review of all the staged and unstaged changes in the current repo. Do not modify the files. These are the instructions that it's given it. So ideally what's happening here is that my main agent's context window is being preserved. So anything that this security review agent is doing is actually ending up in its own sub agent context window. So again when it comes to working with coding agents you do want to exercise the context engineering principles where you want to maintain a very high quality of your context window and this design of working with sub agents implements that. So I have a slide that will give us a better understanding of what's happening here.

[00:58:37]So I've given the main agent the instruction to run a security review but the main agent won't be the one to do that review. It's going to invoke a built-in security agent which will run in its own context window and then it's just going to send the summary back to the main agent. So if you remember um you see that the the the review is done.

[00:59:03]Now when it comes to the context um window you can see that we're still at 35%. This is where my context window was before I started this task. And then the security review agent worked. So you can see this blue small blue entries here.

[00:59:20]So this is context consumption for the um code review agent and that's not like being placed or tracked into my main context window. So that's just something I wanted to call out. The review was completed and we have some issues that have surfaced. So we have two issues.

[00:59:43]Now you can review them or we have some recommended actions here. So the agent is asking how should I proceed with this findings. I will just ask it to fix all the issues the two issues that have come up. And you'll now see that our context window should now see some activity. Now this is the main agent. It's going to work through these issues that have been surfaced and then it's going to hopefully um request a reanalysis or a re-review from the security reviewer. So, hope that makes sense in terms of how the main agent interacts with sub agents and the impact that has um on your context window.

[01:00:24]Again, just looking through the chats.

[01:00:36]Yeah. So, I see some conversations about impeccable. Yes, these are just some um design patterns for your agents to follow.

[01:00:49]Um, right. So, there's a question here. So, I think this is in reference to our second task, which is the the website.

[01:01:04]Looks like it's done. Uh, but the question is you don't need to give uh give it a primary color or a secondary color. So you can I mean you can most definitely when you're invoking the impeccable skill if you have your own design plan you can do that but if you just because the command we used was impeccable um craft. So this is the first option that we went with impeccable craft. So this basically allows the agent to work with the skill and just come up with some recommended design patterns. So yes, I would have gone on and said that hey, I want to work with this and this color palette. So just um ground your plan around it and it should have easily followed that. But in this case, I wanted to see again testing Luna's capability here to just do some design work and see how how how it was able to manage that task.

[01:02:06]But that's a that's a good question. So let's inspect what the agent did here.

[01:02:11]Um yeah, in terms of the context window filling up, I think it's filling up the context window quite fast. If you want a breakdown of where everything is allocated, uh remember the very top, you can click on this session. So just click on the session and then you should be able to see the distribution of your tokens.

[01:02:33]So you can see um again we do have 4 million input tokens and then we have 4 million cached tokens. So the element of token caching here and in case you're not familiar with that token caching is basically a mechanism that um allows this model to just cache some of the prompts that the tokens that get repeatedly sent to the model. So for instance um in this case we just sent like two two conversation ts and since how this agents work is that it basically sends over your conversation history as part of the input most of that conversation history is usually the same. So like the system message and the first turns that you had it's the exact same set of tokens. So the first time you send them to the model, you pay for those as brand new tokens that the model is seeing for the first time. But then when you get to let's say turn number 70, so you've worked on a very long session and you've gotten to turn number 70, ideally from turn number 69 and everything before that, these are tokens that the model has seen before.

[01:03:47]So it's able to cash them and you don't pay like the full amount for your cashed token. So you can see that you'll get that distribution uh of how many tokens are you know like how many tokens are actually cached.

[01:04:02]Then for your context window you of course see the percentage for your system prompt your system tools MCP tools the messages you can see right now are the ones that are consuming so much of my context window and of course you now see the number of AI credits that you have consumed for that particular agent session.

[01:04:22]Okay. Okay. So, let's just look at what the agent has come up with. I'm going to launch this in a new window. So, let me switch over to Chrome.

[01:04:39]Um, so, okay. Wow. This something. Let me see what you guys What do you think of of this first draft?

[01:04:49]It's definitely better than what I've seen from GPT models before. I will give it that it's definitely better. Maybe not quite there yet, but there definitely is an improvement. Um, I love that it was able to try and create some assets, some visual assets that's new. I gave it access to a a skill called remote. Again, these are just so I feel like um the the tool set developer tool sets have really really changed like your tool set right now is more of MCP servers and agent skills and less of like actual tools you interact with. So um I I actually gave it access to remote which it can allow agents to programmatically create their own assets. It can create a video, create screenshots, posters. So, that's something I gave it access to.

[01:05:48]And where is my window?

[01:05:55]Oops. Uh, come on. Okay. Yeah. So, okay. Interesting comment. Uh, yeah, the red is too much and it's too heavy on your eyes. Uh yeah, but I think it's it's actually something like it's not it doesn't follow the common patterns that we see from a generated um UIS at least in my experience in terms of the overall flow and design.

[01:06:28][clears throat] I kind of like this for GP model. This is actually a significant improvement.

[01:06:34]Um yeah, the red is too much. I see it now.

[01:06:38]It's it's too much. It's an interesting choice. Um I What's interesting is that it chose to completely ignore my logo.

[01:06:46]So like I have a logo for key mask and it has access to that logo. So it completely decided to ignore that and came up with its own um logo just again too much of the red.

[01:06:59]But yeah, I think let's see in terms of how the page the scrolling through the page.

[01:07:06]Okay, makes sense.

[01:07:10]Yeah. So, it's it's something. I think it's something. Let me let me know what you guys are think on the chat. But yeah, my my observation is that it's definitely better from what I've seen before from previous GPT models, but not quite there yet. I think that's from from this experience.

[01:07:30]Okay. Um, we are out of time, but let me spend the remaining few minutes just to focus on the questions that I see on the chat.

[01:07:46]Okay. Um, can co-pilot Okay. So, S what I'm trying to read your comment here.

[01:07:51]Can copilot actually create a descriptive summary or explanation of each command which we can learn explicitly to have better understanding using the tools.

[01:08:03]H so summary of each um yeah I don't think I understand that question s probably try rephrasing the question. Um not sure what you mean by copilot creating an explanation of each command.

[01:08:19]Do you mean like having Copilot explain every command that it runs throughout the chat session or is it something different that you're referring to?

[01:08:28]Maybe you could just try and rephrase that question a little bit.

[01:08:40]Um, they work with green checks and not blue. [clears throat] I like blue checks more. Okay.

[01:08:51]Just reading through the chat.

[01:09:00]All right. So, Muhammad is asking um how secure session date is our session is data processed in the cloud or local.

[01:09:10]That's a fantastic question. So everything that you've seen from this demo, like all these different sessions, we just worked with two, but you can already see how this can be like up to 10 different sessions running in parallel. They're all stored in a local folder. It's actually the local copilot folder where all the different work trees are stored. So if I don't know if this will show me that but when we just created a new session like when you create a new session um let's do that together and I say hi then you can see that it will give me the uh like the path. So this goes into my local computer it goes to the copilot folder under repositories. That's where you'll see the list of work trees that have been created. This only gets to the cloud when you push your code on GitHub.

[01:10:08]But everything else you working with the agents whether GitHub hosted agents or those that uh you've you've connected to your model providers all these agent sessions are running locally. So that's uh that's hope that hopefully answers your question.

[01:10:30]Yeah. So great comment here. The problem with a huge context window is that the information in the middle is getting lost and only information at the top and bottom is going to be remembered and used. So this is a problem problem for the agent. That's a valid point and um yeah we can actually talk a bit more about that. So I'm trying to pronounce your name but I don't think that's your actual name. Anyways, our user from Twitch here says and brings up a very valid point. the concept of lost in the middle. When you're dealing with your context window, your agent primar primarily gets to uh remember what you have at the very top like the system message all the instructions that get sent and then it remembers or it refers to like your your last your recent messages. Anything in the middle kind of gets lost in in the process. So a solution for that is compaction where if you see your context window because you're actively monitoring your context window. If you see your context window getting to the percentages of 70 let's say 70% and above then that's a hot area zone. You don't want your context window to be occupied to 70% level. And so what you can do and you can probably practically try this from the website project.

[01:11:54]So you can see this used up 60% of our context window. Now to the point that is being raised here, there's a builtin / compact um command that you can actually use and this is just going to ask the model to summarize everything that it has done so far. So assume you have a very long agent session. You are at turn number 90. Your context window is filling up.

[01:12:20]Then you can use the slash compact command here. Let's just do that. And the agent is going to summarize what it has done and then just reset the context window with that summary just that short summary of what it has done so far. And so we will see that that is what will end up happening. Our compaction is in progress. And then it should basically just reset this context window. And so you can see that it has now moved from 60% to 0% with just a summary of what the agent has done so far.

[01:12:57]Now that is useful when it comes to again what you've just brought up in terms of the quality of the agent. But I would also want to caution you that in as much as compaction here can significantly reduce the number of tokens that are used in that agent session, there's also an associated risk and the trade-off there is information loss. So yes, we have compacted, we have reduced what we've done with the agent to just a short paragraph. So that means in the process we have lost some information that now is accessible to the agent as context. And so what you'll find is that if through um through that compaction process if we dropped very critical information that was relevant to the task that we actually gave the agent then this can ultimately lead to agent agent misses. Right? So you will find that that that like the the tokens that you are aiming to save are actually being cancelled out by a reduction in quality. So you use compaction sparingly. Don't just um use it when you've not used up like a good percentage of your context window. But again, you also want to think about how you're managing your compaction because there is that associated risk of as soon as you compact your conversation. Yes, there's a level of information loss and if in that information loss, you lose very critical information for the agent, then you will see a regression in the quality of your agents. So that's a really good point that you brought up, but I would also want to just caution um using compaction. Again, if you you're heating 80 90% the quality regardless the quality of your of your agent will reduce, but again you also want to use compaction very very sparingly.

[01:15:00]Um, a LinkedIn user is asking, "Can security checks be forced at the enterprise level?"

[01:15:07]at the enterprise level. Um, probably this is not scoped to the demos that we've done today. For today, we just tested like the local builtin security review, the local built-in security review sub agent. But yes, we do have like code reviews running across different stages.

[01:15:29]I'm not sure if that's the question you are asking, but yeah, you can like in this case, we've just made some changes with the agent. I want to have like either a review that is focused on security or I want to have a local review that is just generic. So again, I can invoke the review agent here to just do like a review of the changes we made locally. after I push the changes, you can have like your code review set up on your repo which will do like or enforce an additional check layer of of of review. So you can always set up multiple layers just to ensure that you're getting like the best quality of code um eventually that hits your peer.

[01:16:10]So I'm not sure if that actually answers the question you're asking but let me know.

[01:16:15]Um, okay. I see no more questions, but just to summarize, um, we went through the change log. There's a lot of new stuff that you can probably look at. And then we've just tested two models from the GP56 family. We tested Terror just to see how it handles, like a simple bug um, investigation and fix. It worked fairly okay. um in in my opinion. Then in terms of the context window, I think it manages its context pretty well. I mean for the amount of work we've done, we only got to like 36%. Whereas for Luna, I actually found like it filled up its context window um quite fast. In terms of token caching, we are getting we're seeing almost like 100% token caching here. So I feel like that's actually well managed. Something I'll just call out before we leave is now in this session if I choose to change my model or change the reasoning effort then that resets that cash completely.

[01:17:26]So that means the entire conversation that will be resent to the new model I will switch to I will just pay for those tokens um like I'll just pay again for them. [clears throat] So the full value of leveraging token caching is to ensure that before you start your session, just lock in the model you want to use, lock in the reasoning effort and then just run that session with the same model without having to switch the different between the different [laughter] models just to preserve and avoid breaking your cache.

[01:17:58]Okay. Um so that's it. Uh again, we've done all these from the co-pilot app. If you've not tried it, this is probably a good time to try it out. The link I shared on the chat is um is gh.io/app, but I'm going to put it on the chat again. Uh but we also have a learning series that actually kicks off today. So if you want to learn about GitHub Copilot app, we have a learning series that kicks off today. It will span several weeks in different languages. So today in about 3 4 hours we'll have the English version of a beginner session on the GitHub copilot app that we've been working with. So that's a live stream that you probably want to uh register and join if you have the time. I have put the link for you on the chat and then in the coming weeks we have it being streamed in different languages.

[01:18:58]We have one in Korean next week, Spanish, Portuguese. So you can tune in and just learn all about the GitHub copilot app that you have seen us use throughout this session.

[01:19:10]All right. Um that's it. Thank you all so much for joining. I don't see any new questions.

[01:19:16]So probably this is a good place to stop. But yeah, happy coding. Experiment with the tools. Please share your feedback. if you have features on the app that work well, don't work well, anything that you'd like to see. For instance, in this agent um session, uh in this like insights view, if there are some insights that you'd want to see from your agents, feel free to to share your feedback. Um and the team is really responsive in terms of trying to improve these tools for everyone, both you and your agents. So, thanks for tuning in. H enjoy the rest of your days. I will see you hopefully see you all next week. But if you want to join another version of this live stream later in the day, just keep watching the GitHub channel. We'll have two more live streams today across different time zones. So, you're free to join. Thank you so much and have a good and lovely evening. Bye-bye.

[01:20:49]>> [music] [music] >> Nat.

[01:21:03][music]