OpenAI Codex Hackathon Sydney Final Judging

追加: 2026-05-06

[00:00:02]Welcome back. This is the finals for the OpenAI Codex hackathon Sydney. We've had an intense day of building with Codex jamming GPD 5.5, GPD image 2, the Codex app. Right now, for those online, I don't know if you can see the room here, but it is packed. We had a whole day of demos. Fantastic stuff. There's a lot of pizza, a lot of drinks. The energy is great. People in the room, let's tell the folks online how much fun we had today. Woo!

[00:00:33]So let's dive right in. Today we have our judges for the final round and I'll introduce them and I'll also invite them to come up to say a few words and introduce themselves and what they're expecting today. So we have Andrew from Remisen, Elbert from Folklor, Annie from Build Club, Joshua from Liinal, Murray from UTS startups, myself from OpenAI Codex, and Tyler Codex OpenAI. Tyler, are you in the room by any chance? All right, he's at the back. So maybe I'll invite Andrew and then we just pass the mic in turn. Just introduce yourself, what you're building and what's one thing in AI that's really exciting you.

[00:01:06]Okay.

[00:01:08]Thank you Andrew.

[00:01:10]Thank you. My name is Andrew. I'm a partner at Rambisan. So an early stage VC. So we're backing founders from day zero. Everything from software software meets atoms. Uh weird and wonderful things. Um I'm an ex algorithmic low latency trader. And I think what I'm excited by is how this is like democratizing and learning to do whole different spaces. Things from pesticides, a tech, things that you would take a long time to see if something is even possible. It's possible today. And so super excited to see what everyone's been cooking today for eight hours if it's the best one, Gabriel. But um yeah, super excited.

[00:01:45]Thanks.

[00:01:49]>> Hi everyone. I'm Albert Bilinko and I am a partner at Folklor Ventures. So we are an early stage venture capital fund here in Australia. We've invested about three $200 million so far in vent in startups like some of the ones that are cooking in this room hopefully. And uh surprisingly I am one of the few VCs that has not been disrupted yet. I still have a job um which I'm very excited by.

[00:02:13]Uh and a thing I'm very very excited by with codeex and some of the other innovations is the you know the level of things that are being created now that were never possible before. So we all have these annoying little problems that uh you know little things like maybe indirect taxes for example with our portfolio companies fear and they're actually finding solutions today. So yeah I'm really loving that.

[00:02:40]Cool.

[00:02:45]Hey guys. Um I'm Annie, the founder of Build Club. Um with Build Club, we kind of do two things. So um we run one of the largest AI communities in the world.

[00:02:55]So we have more than 60 um like cities in the world which we're active in and kind of operate like a um like not for-p profofit. Um and then we're also a seed stage startup. Um so our team works on a product called Solaris where we host like the learning academy for example for Manis. Um and we also work with enterprises to help them upskill with the latest AI tools. Um, so something I'm like super passionate about and like excited by with AI is definitely how AI is like disrupting um, especially how like nontechnical people like what they do for their work and also for software engineers um, how it empowers them to like 10x or 100x now. Yeah. Thanks.

[00:03:41]>> All right. Hi guys. So I'm Joshua from Liinal. Uh, we're the venture creation arm of Tamasic. So we work with founders and invested them as well to build whatever they want and to go as far as they want all the way. And I'm just excited to be in the room. I think thanks Thomas and Gabriel for the invitation. And you know I'm just so blown away by the talent in Australia and the creativity as well. I think we do global investments but I think so far and just not to praise the room but Australia has been one of the countries that's blown us away from the talent not just in building but also in business.

[00:04:14]And that's a rare combination and we've just found so many great founders here.

[00:04:18]So I'm just excited to see what's going to be presented today. Thanks everyone.

[00:04:26]Thank you very much. I'm going to make this really quick because I know what you want to actually see. I'm Mari Herps. I'm the director of entrepreneurship and it's been wonderful to have you here with UTS startups today. Can we have a round [snorts] of applause for the UTS startups team that's been helping out around the Thank you very much everyone. Um, I'll also say there's one more judge that we didn't get into the slides, but please welcome Mark Peshy from our side as well.

[00:04:49]>> Thank you, Murray. Uh, been fantastic to just get the vibe tonight and to see so many really good ideas. Uh, I have to tell you, I have been helping the startup community in Sydney get going for what, 15 years now? Oh, Jesus, a long time. And it is amazing to see it firing on all cylinders like it is tonight. So, good luck.

[00:05:09]Thank you.

[00:05:11]and Tyler.

[00:05:12]Last but not least, definitely not last.

[00:05:16]All right. Uh, hi folks. I'll stuck in the back. Um, nice to meet you. My name is Tyler. I am what they call a Codex deployment engineer. Um, I want to kind of show the uh the color of the color of the shirts here cuz if Gabe is like the white uh the the light side of Codex, I am the dark side. I I work with like businesses trying to bring Codex to where uh where they work. Um, so uh nice to meet you all. Um, I'm also a multi- multi- um token multi-billionaire. So, uh, please come talk to me afterwards.

[00:05:45]All right, let's go. All right, finals.

[00:05:54]So, just to recap what the top five will be winning rather three minutes. They'll win the opportunity to demo for three minutes and answer questions for two minutes.

[00:06:06]And they'll be receiving every member in the top five teams will receive one year of chat GPD pro. So go forth token max codeex max. In third place they'll receive 25 grand in USD of open AI API credits 50 grand for second place and for first place they will be receiving 75 grand in API credits. So go forth and token max.

[00:06:33]So, who's ready to hear who's in the top five?

[00:06:42]So, in no particular order, we have Education Unlimited.

[00:06:49]>> Are you in the room? Okay, I'll take it as you. So, if you're in the room, I can't tell. You just got to cheer really loud because I assume you're excited to be in the top five. Okay, next.

[00:07:01]Hemish Boltitude. [laughter] All right.

[00:07:10]Next, Roy in the room. Are you in the room? Roy.

[00:07:16]Okay.

[00:07:20]Next, Chain Breaker.

[00:07:25]>> All right. They are like cheering up there. You know that they excited. And last but not least, we have EI Wizards.

[00:07:39]>> Okay, so now can all the top five teams get one representative who is going to be presenting with their laptop to find Arafat, our Codex ambassador for Sydney.

[00:07:48]He's over here in a white t-shirt. He will help arrange for the Zoom call. So you'll be using your laptop to demo. You have three minutes. You don't have to use the video you prepared earlier. If you want to do slides, sure, but I don't recommend it. Go straight into the demo.

[00:08:02]Let the demo tell a story. So, can I trouble the top five to come forward?

[00:08:05]Come down, please. Come down. Okay.

[00:08:10]So, I know this may be disappointing like, oh, uh, no. Find Arafat over here.

[00:08:17]Yeah. Not me. Not me. Yeah. So, Arafat, our codex ambassador Sydney will help get you get set up. So I know maybe like damn like I worked really hard. Codex worked really really hard. I didn't make the top five. As I said again this was incredibly impressive array of projects.

[00:08:36]This is absolutely insane. And it doesn't end here. I mean the hackathon was just an opportunity to try something new, an opportunity to test a new idea, an opportunity to make a new friend, maybe a future co-founder and it just starts from today. were quite intentional and also asking you to prepare a two-minute video. Now you've done the leg work of sort of packaging the story, sharing the demo, publish it, share it, put it on LinkedIn, put it on X, put it on Reddit. We work to the R Codex community to do this. Build in public. That's the best way to get your story out there. It's a world where so much things are happening and when you build in public, it's great. So feel free to tag us, tag the judges, tag UTS startups, tag OpenAI devs, and we will help share the story. It's an incredibly impressive array of projects. At the end of the day, you know the problems best.

[00:09:24]You know the technology and you're going to build the future. Developers are incredibly important in our mission for building AGI that benefits all of humanity. We really mean it. We're going to build products. We build platforms for you to sort of build upon. So, it's been an incredible time to be in a builder. So, the presentation order will be as listed here. Education unlimited and uh is the team ready and so forth.

[00:09:47]Hamish bullish roy chain breaker and so forth. So uh once you're ready can give me a thumbs up. Uh education unlimited.

[00:09:53]You good? Ready. All right. So let's get dive right in. So uh first up Education Unlimited.

[00:10:05]>> Thank you. Maybe to start, can I have a show of hands of who has kids in the room or relatives that have kids or nephews or nieces? Yeah. How hard is it to get them off YouTube when they're in front of a computer? Is it is it hard?

[00:10:17]Is it difficult? Is that what they do?

[00:10:18]They consume content. They don't learn.

[00:10:21]They don't learn to code. They don't learn skills. They just consume blindly.

[00:10:25]So, a big part of what I'm focusing on with this platform with Primark is helping to take AI to the next generation to make them engage in a way that is genuinely interesting to them.

[00:10:37]It focuses on their interests, on their passions. It frames the things they have to learn. you know, there's a curriculum behind the scenes that they have to get to the end of school knowing, but unless we give it to them in a form that they're willing to get excited about, they're willing to get interested in, then it's not going to work. So, the primer platform looks simple as you see it here. Yes, you know, it asks you at the start what your interests are, what your student it cares about, what their passions are, what their biography is, how old they are, and yes, it does frame lessons in this form here where it gives them a story to suck them in to the lesson they'll be learning. It gives them an idea as to how to think about the problem they've got. You know, yes, this is a physics problem, but it's framed it in this form. You know, my son is interested in basketball. He's interested in uh the natural life, particularly marine biology, but it's telling us right here in a very discoverable form how gravity works, how velocity works, how the basic principles of physics work. So, it's able to keep us engaged in things that we care about.

[00:11:42]But behind the scenes is where the complexity happens in a platform like this. You cannot just present endless lessons to a student getting them to the end of this. No matter how engaging it is, no matter how amazing things like GPT image 2 are, yes, you can grade them, but they might get through three or four of these and that will be the end. They'll walk away. They'll go back to YouTube. Behind the scenes here, the system itself is learning everything it needs to know about that student. So, what we're looking at here is some of the research I've been looking at in the last couple of years in bitemporal graph memory. And what that is is a system that observes and learns as someone engages with the platform. It draws connections between me as an individual, the things I'm doing inside the platform. You know, it tells me what kind of learning style I prefer, what I'm interested in, the things that I need to do to make sure that it's not just a thing that's one and done. It's not someone that's there for 30 minutes.

[00:12:36]It's someone that's there permanently and they keep coming back because the platform adapts with them.

[00:12:42]And of course, on the other side, you've got parents and teachers and they need to know, is this learner actually doing something meaningful? Are they learning what they need to learn? Are they understanding the content? So, this platform has actually grabbed the Australian high school curriculum uh off the uh the New South Wales website.

[00:13:01]>> Sorry. [laughter] >> Any questions from the judges?

[00:13:14][laughter] >> Uh yeah. So are you able to just tell us a little bit more about you got cut off obviously but a little bit more about the usage and then also monetization as well.

[00:13:25]>> Yeah absolutely. So >> if you look at the education sector even the digital education sector it's super super expensive like I'm sure a lot of people that have kids would have bought practice tests for things like nap planan and the selective schools test.

[00:13:38]So there's clearly a propensity to pay a willingness to pay. But if you can demonstrate real results and that's kind of what this system is here. If the parent can see, you know, this isn't just something that they're sitting down and doing the equivalent of YouTube with, the student actually comes out with something meaningful, then they're likely to stay, the lifetime value of that goes up because ultimately they see that they're getting value. They can see the result of that much more so than they could with a platform that doesn't use AI that just looks at did they get the question right or wrong.

[00:14:10]>> Can I just ask how what did codeex enable you to build here? like what was unique about the codeex angle um to help you build today?

[00:14:19]>> Yeah. Yeah. Couple of things, right? So, you know, over the last couple of years, we've obviously seen a huge evolution in model capability. So, the ability to take an understanding of a student or a learner and to frame something in a structured way that can be presented in a visually attractive way, but also actually help you to progress linearly through something to the point where you get to that conclusion. The can they answer the questions about the content?

[00:14:44]I don't think that's something that you could have even done with the generation of models from last year and particularly things like what we saw with GPT image 2 on uh this thing that we saw before. That's really only been possible in the last couple of weeks.

[00:14:57]Now, yes, we could generate images, but could you generate something that's actually correct and that you could rely on to communicate an idea and mathematics without misspelling words or misrepresenting labels on a diagram? uh now we can and codeex alongside all of the AI technology behind it is what makes this a conducive platform.

[00:15:21]>> Fantastic.

[00:15:21]>> Thank you.

[00:15:26]>> All right, next. Um sorry, I forgot your team name.

[00:15:29]>> Roy. Okay.

[00:15:31]>> Hello everyone. All right, let me just connect to Zoom. So the time the timer starts when you first adjust his first word, his first token.

[00:15:40]>> Oh, can you disconnect from the screen?

[00:15:42]Oh, will someone allow that?

[00:15:46]>> Eating into my time, mate.

[00:15:57]>> No worries. I'll I'll just start talking. So um since uh since starting to use the Codeex desktop app, I find myself using voice for nearly everything. Um, but the one thing I can't use voice for yet is uh PR reviews. And PR reviews happen to be like the the bottleneck to writing production quality code right now. Um, I I work at a big company with an enormous codebase and and we spend a lot of time now on PRs. So I wanted to I wanted to explore what a voice oriented agentic uh PR reviewing uh solution would look like. I'd really love to um I'd really love to have something like this in uh in Codex itself. Um, so I can get a live demo talking to a voice agent. So here we are. Um, so here I'm I've loaded up a PR. I can see the files that are changed on the left. In the middle is um is the is the code changed. I can I can look around the code, you know, like I normally would. Um, and on the right hand side is sort of like my AI work area. And I'm going to start the voice.

[00:16:56]All right. Could you please uh give me a summary of uh what's changing in this >> PR?

[00:17:08]>> Can you speak up a bit? Say it again, please.

[00:17:12]>> Sure. This introduces a full GitHub review.

[00:17:15]>> Cool. So I can it can explain it to you.

[00:17:17]I actually prefer to read anyway. Um, so speaking of reading, um, you know, can you, uh, can you maybe show me a mermaid diagram for, uh, for this PR so I can understand what's going on here? So what's happened here is, um, what's happened here is, there we go. What's happened here is, um, I've actually dele so this has got an agent running in the browser right now that has a set of tools that allows it to answer questions from the context it has, but also delegate out to Codeex um, app server on the back end for deeper analysis. Uh it this mermaid diagram I prepared earlier and it will look at you know its current threads and see if it can answer from from current thread. Um so here's a mermaid diagram.

[00:17:51]Great. That can help me get a good understanding of this PR. Um something else I prepared earlier was like testing gaps. So please could you please tell me if there are any testing gaps in this >> show me the report for the testing gaps.

[00:18:10]There's a few Can you take me to line of code that has a testing cap.

[00:18:18]So, this allows you to actually navigate around the PR as well. Um, so obviously you can delegate out to Codeex to do some research. It'll tell you where you should look and then you can actually go and look. Please add a PR com.

[00:18:33]And then it can actually author the PR comment for you as well.

[00:18:38]I don't know if it did that. There you go. See, it's just added it there. Then finally, when you're ready, you can actually ask add a PR description saying the tools that I found and then submit it, please. Then you can actually go and submit it. So it's going to be uh entirely handsree.

[00:18:52]Am I out of time yet? Do I need to keep doing this? [laughter] >> Second.

[00:18:56]>> All right. Um so what else can it do? Um so uh a thing that I haven't demoed here and don't want to demo live is it can actually propose changes as well. So you know if you know if you've got a unit test that's missing, you can actually just say hey you know propose the actual unit test. Um, and it'll actually integrate and push that up to GitHub so they can just hit accept changes on on that end too. Um, I also wanted to take it as far as like actually writing coding code with this so you could like work on your own PR to update it, but I didn't get that far.

[00:19:24]>> Great question, >> Royce. That's really interesting. I'm wondering two things. First off, does it change the way you're working with the code now that you're able to talk to it and it responds? And second thing is, have you thought about people who might be sight impaired also being able to use the tool?

[00:19:48]>> So the f the answer to the first question is absolutely. Um I find typing to be extremely frictionful because when I type, I have to think about how I'm actually going to word what I'm saying, whereas I can just ramble on for hours and hours and it makes sense of of what I'm saying. Um, so often like my when I'm reviewing an actual PR from from Codeex, often I'm just going from top to bottom with it listening and I'm just like, "Oh, you know what? How does this work? Oh man, what what are you doing here? Oh, okay. I get it." And and it just ignores all of that stuff and extracts from that the signal and then it goes and works for 10 minutes, fixes the problems and I'm good. So absolutely change my workflow and I think this could change something like this could change our review workflow too. Um, then vision impaired people, I mean surely surely it's going to be really really helpful for them. Um, and uh, I I personally haven't thought about it that much. Uh, I'm just busy getting stuff done myself. But yeah, I imagine this could be a big bo for them, especially since like I didn't demo it here. It can actually explain this stuff to you really well.

[00:20:44]>> Does this become a gnarly orth problem, you know, who who actually is talking?

[00:20:50]>> Uh, I guess so. I mean, like the intent is not that you run through a demo and just like, oh yeah, that PR comment looks good. You you should actually be understanding it. And that's why I wanted to demonstrate the the mermaid diagram here as well. It's really really important to actually understand this code and understand how it fits in with the rest of the architecture. Something I didn't demo here is that actually does give you a bit of like an architectural coherence review as well. And um one thing I wanted to build was like here's here's how you should actually approach um uh like reviewing this. Um this is actually configurable. So what I would imagine in like a place where I work at Dropbox, we have might might have many teams who would like insert their own like hey if you're working on identity you should consider this and and like in inject that into your panel as well to give you advice on how to review it or maybe context that you need like a like a reviewers MD instead of an agent MD.

[00:21:35]>> Does the workflow change if the surface area of the PR that's changing uh you know change if it's very risky surface area of the code? Have you thought about how to handle that in voice?

[00:21:46]>> Not really. I think um I think you probably want some like warnings about like you know like severity come up in here. So like a big red sign if there's something particularly risky. Um this thing could totally like have like a P 0, P1, P2 sort of grading of of the issues that it finds.

[00:22:04]>> Sorry, one last question. I know you built this for PRs. What's the nearest adjacency that this would also start to work well for?

[00:22:12]>> Do you mean in like coding?

[00:22:15]Um, I mean, you could use something like this for coding itself. I mean, like at the moment, coding is still very turnbased when you're talking, right?

[00:22:22]Um, I would love something where I could watch what it's doing go by. I'm often reading the code as it's going by as well. And to be able to talk and like to either steer and say, "Wait, stop." And have it know that that that it should steer or um or at least just like accumulate notes and and ingest that signal as I'm talking. Um, that would be super ideal. Instead, at the moment, I sort of have to wait and then and then I talk for 5 minutes and then it goes for for 15 minutes later. Um, and then like I imagine something like this could also be great for reviewing like actual text documents. So I'm a principal engineer.

[00:22:50]I spend a lot of time working on documents and um something like this could be just great at like reviewing, you know, anything like driving confluence or something like that.

[00:23:00]>> Thank you.

[00:23:04]>> Oh, okay. All right. Nexus Hamish will hamish Boltitude.

[00:23:13]>> [applause] >> Yeah. Can you come down and set up?

[00:23:16]>> Yeah. Yeah. Just come down.

[00:23:20]>> Awesome. Hello everyone. Uh, so I've got a more It's Look, I respect everyone that's doing these really cool PR workflows today. Mine's more artistic.

[00:23:28]So, um, I'm not going to play through the entire Pokémon video game in front of you, so I'm just going to play my video and I hope that's okay. Uh, so yeah. Hey, I'm Hamish. Um, I worked at camp for 5 years. I'm currently at a AI agentic coding uh startup called AMP.

[00:23:42]We're one of the largest coding uh harnesses in the world. So yeah, really excited to be here today. So you know how many here, you know, have played Pokémon in the past. Put up your hands.

[00:23:52]Yeah. Okay. Okay. You know, half the crowd. Um you would have seen a bit earlier today there was a Doom demo where people could use codecs inside Doom, right? Okay. So Doom Source has been available for a really long time.

[00:24:02]Uh video games, you know, in general don't have their source available.

[00:24:05]Pokemon Emerald and the other game I'm about to demonstrate today, Legend of Zelda. Um, they both were reinvent like had their original machine by code turned back into C. So, what does that mean? We can that means we have the, you know, the source code available to be able to edit and we can do some really fun things with that. So, I'm going to go through ignore, you know, smaller hamish talking in the background. Um, so at the start of the game, you know, we have the ability to actually choose our own Pokemon and this is the Pokemon that goes with us for the rest of the game.

[00:24:32]So, you might notice here, I'm getting some ringing. I'll stand forward a little bit. You might notice here there's an open AI Pokemon, Altman.

[00:24:40]Interesting. Uh, that's actually Sam Alman that I put in the game. Crazy. Uh, you know, there's also, uh, my girlfriend, uh, that I decided to put in. I wanted to put in Gabe. I'm sorry, Gabe. Uh, and then also my dog once we get over to that. So, yeah, I've been able to inject those into the game.

[00:24:56]That's good fun. Um, so I choose Alman.

[00:24:59]Uh, he decides to attack the dog. Let's skip that part. Uh, anyways, so I've added a new character in the game actually. And here's this little character here. Um, this is actually a custom character that I created myself that's linked up to codeex. And so what this means is that, you know, while the game's running, we have some RAM. In the RAM, you know, all of the game state is held. You know, where the character is, what Pokémon you have. I'm actually able to talk to this character and it's linked up to codeex. So I can give it some actions. For example, I can say hi uh or hello. When I run this um, you know, we reach out to Codeex. It takes a little while to think. Codex says, "Hey there, trainer. Ready to start your adventure?" So that's not hard-coded.

[00:25:35]That's actually linked to Codex app server, which is reading the game, like the internal game memory while the game is running. And we're able to pull out that like Codex has been able to disambiguate that, disambiguate that RAM and translate that into a prompt, which is actually really, really hard if you think about it, right? Like this is unbelievable. And what I couldn't do without AI. Um, another example, um, if I type the character one, I immediately get gener like I have a tool. That tool just grants me a very high level character. That's cool. Charizard.

[00:26:03]Lastly, I can type in Finn, which just finishes the game. Just skips right to the end. Uh, yeah, there you go. Done.

[00:26:09]Um, so I did that in half the day. I decided I wanted to go a bit further with another game. This is a bit, this is 3D this time, so it's actually quite harder. Um, we don't get all the same game primitives that we want to interact with. Uh, and so this time I created my own or Codex helped me create a keyboard on the screen. I'm able to type into that keyboard. Uh, you know, hi. This time it's a bit harder to type and uh you know we wait for Codex and we hear back from Codex in a sec. It says hi link which is the name of the character.

[00:26:39]>> Cool. That's it.

[00:26:41]Thank you >> questions.

[00:26:48]>> I have a question. um how hard was it or how easy was it for Codex to you know do all of this like emulator ram access and everything?

[00:26:57]>> Yeah, that's a great question. So um the amazing thing is like all the code has you know reasonable file names you know all of the functions are annotated properly. So, you know, there'll be there'll be a function like, you know, debug text box, and that allows me to draw something on the screen or, you know, like there the the hardest part was actually getting coex to create that um I've closed the computer now, but create the keyboard in the last demo, the 3D demo, because that wasn't a primitive that already existed. I was able to reuse some of the Pokemon naming uh like you give your Pokemon a nickname when you catch it. I was able to just inject that in for the Pokemon Emerald version. So, yeah, it's es and flows.

[00:27:29]Yeah.

[00:27:33]So, what do you think this all means for the future of video games then? Like, if you kind of take a broader step, it's really cool what you've done. I loved Zelda growing up.

[00:27:42]>> Yeah, nice same.

[00:27:43]>> You know, if you kind of take a big step back, where does the future lie?

[00:27:47]>> That's a great question. Um, you know, I think a lot of the games that are coming out now, um, I I haven't played Roblox much, but Roblox is one of the biggest games in the world. Um, and a lot of that is like based on user generated content. you know, people can go along, create their own games, create their own stories, and actually sell those for a lot of money. Um, and so I think, you know, whilst this this demo specifically is more, you know, something that I've wanted to do for a long time, and I find a lot of like accomplishment in, like injecting Sam Alman into Pokémon, who would think of that? It's kind of weird.

[00:28:12]Um, but yeah, having having that, you know, I think there's a lot of, you know, creativity in that space, and I I like exploring that. It's a lot of fun.

[00:28:21]>> What's the artifact from today? Is it the engine? Is it the output of the game? Like where where do you go next?

[00:28:28]>> Yeah, good question. So, um the original like the input, let's start with that.

[00:28:32]That's a bunch of C and assembly code.

[00:28:34]Um there is a make file which you run against that code and that generates a like a ROM file or like a like you know there's a program that you can execute in the same way it's like aexe file. Um and then that game is given to a game boy and then that goes from there. So all of the like character data is like packaged into that.

[00:28:48]>> Right.

[00:28:49]>> Yeah.

[00:28:50]>> Very cool. I you know I used to play the emulators you know.

[00:28:53]>> Yeah. Same.

[00:28:53]>> Yeah. Yeah. So yeah, that was running on an emulator on my Mac and reading out the RAM from that. Yeah, cool. Thanks for listening.

[00:29:06]>> All right.

[00:29:09]>> Hey guys, how you going? It's great to be here.

[00:29:12]>> Let me just get set up real quick.

[00:29:23]Cool. Cool.

[00:29:31]So um EI wizards have created a basically accessibility layer for the NDIS. Now why have we chosen to solve uh this particular problem and point codecs at this particular problem is because uh we actually found online that there have been a number of efforts by the NDIS to increase access and visibility and ease of use to their platform and data. So we thought you know why not point Codex to this and try and solve a problem impacting uh real Australians. So what we've came up what we've come up with is effectively an accessibility layer that sits on top of the website and you can see that here. It's called Access Mate.

[00:30:12]So the idea is it's a browser extension that can probably sit on top of any website but has a knowledge source of lots of the data uh per domain. So we basically took a bunch of NDIS data, put it into uh markdown files, sort of open claw memory style that codeex um is able to gp over and reason about different NDIS facts and data and workflows etc. Um we also added in a couple other features. So we added computer use so the agent can scroll the screen and you know show the user different parts of the screen. Um, we added uh what else we added? Access to the DOM, very similar to the codeex app, um, with the annotation features that have recently been released. And what that lets, uh, the access mate do is read the website and see all the information on there and answer questions that a user might actually have. Um, we also have the real-time API, so you can, you know, interact very fast. And we've actually got the uh codeex CLI running as well.

[00:31:20]So you can uh add different elements to the screen on demand dynamically. So I'll just um unmute the mic and take you guys through a little bit of it.

[00:31:32]>> Can you tell me what website this is?

[00:31:36]>> You're on the NDIS website. It's the main place to learn about the National Disability Insurance.

[00:31:42]>> Cool. Uh are you able to scroll down the page for me?

[00:31:48]I've scrolled down the page for you. Let me know if you'd like to go further or if there's something specific you'd like to find.

[00:31:55]>> No, that's great. Um, I can see that there's a eligibility button. Uh, is that a hyperlink? Would you be able to click that for me?

[00:32:07]Great.

[00:32:08]So you can see that I've now been able to navigate and ask questions and learn about different parts of the uh of of access body. So um one thing I might ask on this page is maybe >> sorry I forgot to read on mute. Uh one thing I might ask is could you tell me what's on this page and you know give me a checklist of the main points. Hey, can you tell me the main points from this uh page and put it in a checklist for me?

[00:32:41]So, this is where we have invoked codeex and are able to send it data from the website and get it to dynamically generate a UI element that's overlaid on top of the screen. And [snorts] u we've focused on just a checklist. Um, but really you'd be able to extend it and, you know, add different flashing elements or moving elements or, you know, any sort of accessibility feature you can imagine, whether that's like changing the color format or um, yeah, really whatever you can imagine. And that's how we've integrated codeex into this.

[00:33:16][applause] >> Great presentation. I do want to ask I mean we actually investing quite a lot in assisted living facilities and for the elderly and this is actually fantastic. Uh a question I do have is that especially given that most of the elderly or folks that we think would use it we as we observe mostly do it on like their phones on their iPads for example where like I'm not sure whether this is applicable or not or how do how would you think you can deploy this onto those devices where you may encounter some resistance?

[00:33:47]>> Yeah, I think that's a great question.

[00:33:48]So um I initially I we actually tried to make a website that would render the NDIS inside of it. Um but we faced some technical difficulties doing that and hence had the browser extension.

[00:34:01]However, I think that if if there was an organization that was committed to having this sort of technology accessible to their users, we'd be able to embed that directly on the site and make it a native experience.

[00:34:16]Um really great presentation. Um keen to understand a bit more like out of all the features which were built which one you found like the most technically challenging. Um and then also a bit more about like the underlying tech stack for example for like the browser use what you used for that.

[00:34:33]>> Yeah so we used uh the open API so open AI APIs for everything. Um so that was the computer use the real time and then uh the codeex uh subscriptions um for the codeex use but um yeah that was the main tech stack um we're able to compile a browser extension quite quite quite easily actually it was kind of surprising um uh yeah how do you manage the latency issues and kind of keep it under 100 ms like like think about managing state and latency uh how how would you approach it I guess using was it an issue or you know how you how would you go after that?

[00:35:16]>> Yeah, so there were there have been a couple um open a open open AI uh open source repos that are like agents uh and real time related. So we definitely looked at those and sort of try to incorporate uh that as much as possible.

[00:35:33]Um, but really I think like what matters is using the correct network protocols, stuff like gRPC and taking all that stuff into consideration when you're building a lower latency app.

[00:35:43]>> Um, have you given any thought to the fact that because this is an NDIS website, people might be putting in personal data and how you might be able to keep that secure.

[00:35:52]>> Yeah, definitely. So, um, the way that this works is we actually have a server that can communicate and host all like the LLM related workflows and and queries. So that server is on Azure and um seeing as it's there I think that we'd be able to run potentially some of the open AI um PII reduction uh models as well that I think one came out only like a couple weeks ago. So you know that would be a great integration I think.

[00:36:20]>> Thank you.

[00:36:24]Last but not least, uh, chain breaker.

[00:36:39]Good evening everyone.

[00:36:41]Okay. Hey bro. Um, actually yesterday I was asking for my mom to get an a dog for my house but she rejected. She doesn't want dog actually. But I actually has an startup idea to make a robot dog. You know that cool one in the UTS. Yeah, that yellow one. Yeah, I want that. Hold up. I think I got an idea for the startup to use the light using AI to redirect the point. I actually had an idea to build a bin to move whenever you throw it. It will catch my garbage. I have so many idea. Oh my god. But I don't want to waste time to visualize it in real life.

[00:37:19]So, have you heard of reality MVP?

[00:37:23]>> What's that?

[00:37:27]So, okay, good. Pick up the phone, access it, and start building.

[00:37:34]So, we are the chain breaker. So, we built reality MVP.

[00:37:40]So, you can see uh all we do is just uh take a pictures of this sketchy. When you have a startup idea, it would take the input of sketchy images here and also you can input text and then we use open AI to ask the related question to refine your idea and then with open AI API we're going to mix it uh up and then we're going to generate the GB file in order to create a 3D object. So for founders they won't wasting too much time using blender or other things to mock up everything. And after generating we will be having a uh result here.

[00:38:26]So because the waiting time is too long.

[00:38:28]So we just want to show you how it's going to be in the real life. For the waiting time it's going to be from image and text to go file will be four to five minutes. Uh which is reasonable and after that this is going to be our output here. And if you click into the view in AR file you can see here reality MVP prototype. Now we want to make this robot arm sketchy here. We actually make it into that robot arm. You can see in 3D models preview that and good thing is that you can view in AR.

[00:39:07]>> So just bring the sketchy stuff in your paper to real life.

[00:39:13]Give it a second.

[00:39:19]Wh is it? All right. All right. Here's a little buddy. I can be Tony Stark by this man.

[00:39:27]Look at this. and make it bigger actually. Yeah. Cool. Cool. Right. But honestly, this is not it. Not just stopping at the AR, it also creates many other features such as it's also building a launch page for me. So with the startup idea, you can have a launch page in case you think that oh, it's feasible and you can make a launch pace for using the weight list. So people who are interest can register because it's also a good thing for you to validate your idea to see the demand whether the people's actually wants it or not.

[00:40:14]ask it to build for me.

[00:40:19]>> Yeah. But I mean, is there a limit? In other words, >> yeah. Yeah. Put yourself on the mic.

[00:40:24]>> Yeah. So, for the image, like do you have any idea in your mind?

[00:40:29]>> Build me a Lamborghini >> like in other words, a reasonably complex object with lots of moving parts.

[00:40:37]So we have the limit for the input which is uh 800 words in text and also for the sketchy we just accept just a sketch of the idea and also the AI will ask the question back like how do you want the joints to be for the arm how many fingers all things that to add to the detail what material you want to be what color or so um so I would say that it can build to the medium complex level >> does it adhere to the physical world like physics like the output you know does it understand heat you know everything that norm [clears throat] you know you would you would hope that it could potentially do or >> you mean like uh actually physics >> like the real world yeah >> uh to be honest we using as a serving for people who has a quick idea and want to seize it in real life and want to save time rather than just moving around using blender all the sort of stuff to visualize in real life So that is not the pain point that we trying to solve.

[00:41:37]We're just solving the pain point of trying to visualize in real life.

[00:41:40]>> But that's a good point anyway. So maybe that's our like future directions that we can build up from that and also that's a good things that we can add it to the models that we can generate based on that physical um conditions. I can guess like if it's need uh to like adapt to the heat, it's actually can do with the the metal uh I mean like the materials that needed for that.

[00:42:08]>> Super good demo and nice skit. Um the question I have is like given you can put any image inside is there any filter or any like especially for copyright materials of any sort, right? Do you filter any of those out currently or do you just to your point just take whatever is on a photo and then you I don't know you search on the internet just put it out there like is there any sort of rights protections that you guys have currently built in?

[00:42:34]>> So uh your question is about the copyright because >> copyright or even obscene material right unless you're allowing that as well.

[00:42:40]>> Yeah. So for now we not I mean like actually I think that the models can be like must be trained on those kind of like um resources but um for now we just utilize the power. So we not actually dive that deep into the copyright uh issues. But what I can tell during our test um it's like we just sketching something and it's automatically generate like what we want. So actually it's going to based on what the model learn.

[00:43:17]>> So it's kind of like uh you visualize the MVP like is MVP of MVP like it's not even a physical one like you just want to see in real life and in order to touch copyrighting thing for the startup I think they would iterate it later on.

[00:43:30]Yeah.

[00:43:31]>> Thank you.

[00:43:34]Uh, so I was just wondering like can you go into a little bit more detail on who you think your customer will be? So like obviously it's someone who has an idea, wants to take a photo and visualize it and it's not for example a professional designer who has tools already but like who who could they be? Like what I don't know just like elaborate a bit more because there obviously a lot of people have ideas but you need them to turn it into a reality to be consistent paying customers.

[00:44:01]>> Yes. So you know start mate right? Uh so I have I actually um joined an event of them and I actually met a founder and he is just uh he's not a technical founder and all he his idea is pretty funny is actually make a delivery robots to replace human in Australia post because he's seen in China like how delivery robots are so advanced but actually he's not from technical and he's just telling me that it's so hard to use blender and um I I would say that he's can be uh one of the customer and you know in UTS startup They have Pura link which is the robot inspection for the pipeline. They in their early days I believe that they can use our product to see how it fits in the pipeline, how it looks like in real life and I believe those are uh our customer.

[00:44:50]>> Thank you.

[00:44:51]>> Thank you.

[00:44:54]>> So one more round of applause for the finalist. I think it was a really amazing show. it had to like be spontaneous and present on the SW.

[00:45:03][applause] So, now we're going to transition to the judging deliberation. Uh, for everyone else in the room, uh, we have pizza and beer. For those online, I apologize. The pizza apparently is pretty good. Uh, we'll cut the feed right now, but we'll be back uh thereafter to announce the top five. Uh, the winners rather. So, uh, thank you for dialing in and see you real soon.

[00:45:24]Okay. So, for those in the room, uh enjoy the pizza, enjoy the beer. Uh for the judges, uh maybe we if you're good, we can just deliberate here and All right. We'll come back very shortly.

[00:45:48]Is it very obvious for you? Just like pulling it off or I everybody use codeex to do the data but like from the it's like [music] you want us to >> uh you can do it.

[00:45:59]>> Okay, I'll answer.

[00:46:06]Uh yeah, if you can do it now.

[00:46:10]Yes, [music] >> I email you. But I can do it.

[00:46:19][music] [music] >> Not too bad. Not too bad. Pretty good. I mean genuinely like [music] okay sorry [music] m [music] All right.

[00:46:56][music] [music] >> Send it on the back. [music] [music] There it is.

[00:47:30]>> Yeah, I did a pretty good job.

[00:47:33][music] [music] >> Yeah.

[00:47:48]>> [music] >> Okay.

[00:48:03][music] Thank you, sir.

[00:48:07][music] >> [music] >> Yes, I can.

[00:48:18]>> Why is it in Japanese?

[00:48:19]>> Why is it in Japanese?

[00:48:21][music] >> Yes.

[00:48:25]You don't [music] [music] [music] have to open in Japan. Oh my god.

[00:48:49]Like very different. [music] very subjective [music] [music] look at the data problem starts [music] Yes.

[00:49:23][music] >> [music] >> I think last guy [music] is I just [music] [music] >> Where are the people that were there like one person that came? Was that like a [music] [music] Gabriel?

[00:50:30]>> [music] [music] [music] [music] >> We're also trying to challenge him to get to um absolutely >> [music] >> like something touch.

[00:51:03][music] All right.

[00:51:10][music] A lot of people use like >> [music] [music] >> All right.

[00:52:07]I saw you trying to get here. I try my eyes.

[00:52:15][music] >> [music] >> The last one like [music] this studio >> [music] >> All right, the results are we normalize it by judge as well to [music] make it more fair.

[00:53:04]>> If you I'm more conservative.

[00:53:07]I >> ask to explain what I mean. Uh first we have the Pokemon one. Second uh the ND and the third [music] one is Royce which is mastery.

[00:53:22][music] I think we probably I think I think [music] the game platform had a lot more resonance on your generation than it did with I think that [music] the the Game Boy integration had a lot more resonance with your generation than it does with I don't see a lot of value in it but we knew it was going to play well [music] like we get the actual game like you can create and tap into the low level with a compiler and create all these yourself [music] if I may be fundamentally the theme is about doing something that you couldn't do without codeex. You could build ND service building fundamentally best picture.

[00:54:38][music] Very happy [music] that >> [music] >> Okay.

[00:55:08][music] Okay. 1 2 3 1 [music] 2 3 [singing] >> [music] >> Uh yes. Um yeah. Yeah. So I I kind [music] of do the whole cycle. So like everything. Um [music] and then we also just like do a lot of testing because we want to see how far we can push. Um [music] just like we're part of business. Um so that's where all the token spend come like we do a lot of experiment.

[00:55:57]Yeah. Yeah. So probably probably one of the biggest cost centers like in the whole company. [laughter] >> Well Peter is not that high actually.

[00:56:17]I'm like number 12 in the whole company.

[00:56:20]There's like above me people above me. You just leave your mouth and you're like, "How many people even have like uh funny enough, but I I just have like claws, but uh you can actually um port over the uh the automation aspect like regular CLI you can go into a server have it read [music] everything about your work uh and then kind of do your work [music] >> yeah so it's still running right now like yeah and it taps into my so I can just like leave [music] comments there and yeah so originally I use linear and then linear uh that I maximize my API because I'm a paying customer. [music] They're like you can't do anymore.

[00:57:29]>> I know. So it's like per user like they said you can't increase anymore. Um but then I went to GitHub [music] cuz GitHub seemingly you willing to pay money is okay. Um so I have everything [music] running like the UI aspect of it is actually on GitHub. So I can just leave comments on like the repo and then it will it will actually call like COX and the server and then just start work.

[00:57:52][music] find people who are not cold but for my own like [music] I like it found randomly that uh they sent you emails but it went to the span but then it picked up the fact that it was like a lot of seats like hey like you're missing out like go talk [music] to your account director again and I was like okay and then yeah then we ended up getting it and then I wasn't [music] uh Oh, they they know perfectly.

[00:58:25]>> Hello. Hello.

[00:58:26]>> When when you >> the reasoning models have reasoned, the mixture of experts have experted and we have the results.

[00:58:36]>> We have the results. So this is this has been the OpenAI Codex Hackathon Sydney.

[00:58:44]An amazing crowd. It's been an amazing day. Round of applause again for everyone. [cheering] >> [applause] >> So before we reveal the top three, let's talk about what's next feedback. So we really want to deliver quality experience. We really want to make Codex the best application for you.

[00:59:07]We want to know what works, what doesn't. Please give us your feedback.

[00:59:11]With the feedback, we also prepared some swag, an OpenAI t-shirt, exclusive OpenAI caps to thank you for your time and filling up this feedback. We want to know how we can better support the builder community here. So, we'll flash the QR code at the end as well. Next, Codex ambassadors. So, in the room right now, uh, we have Arafat. Arafat, please come up. Come up, come come up. Arafat is our Codex ambassador for Sydney. So, we have a bunch of Codex ambassadors all around the world, also in Australia, but if you look at the map, it's a bit sparse right now. We have only a few cities and I understand a bunch of people in the room flew in for today's hackathon. Thank you for coming by and if you want to see more of this, let's join us as a Codex ambassador. Drop in a form, let us know. And we would like to bring this to more. We can't do this everywhere. We really wish we could, but with your help, we can. So before we uh move on, I just want to let Arafat share a few words, introduce himself, and how we can sort of work with the community here. Arafat, please.

[01:00:12]>> Thank you. Um yeah everyone um I've hosted a community demo over here. So I'm really excited to launch the OpenAI codeex community today. Um what we are going to do in uh we in a month or so is we will try to host some of the meetups.

[01:00:29]So you most you are most welcome to come over and talk about it like what you are building with codeex. Um we really want to build a space uh which will give every founder everyone over here in the community um to really build things with as latest as GBD 5.5 and we'll try to get you some of the credits courtesy Gabriel. So yeah um feel free to join um so we'll announce it. So if you are a part of this network um we'll be sending an email to all of you um and let you know about the future plans and as soon as it is decided and finalized I'll I'll keep you posted. So yeah you can join us then. Thanks [applause] and before we end I just want to again say a few words of thanks for everyone who made this possible. Uh firstly our judges for coming down on the evening to like spend their time evaluating but and thinking about different projects. So please round of applause for their judges. [applause] Uh next of course the amazing team from UTS that made everything possible.

[01:01:41][applause] So in particular the idea of having it today was Murray's idea. who's been pitching about this for a long time and we know this is special and we're going to do it and we did it again. Thank you M and of course also want to thank my uh my my colleagues from opening Australia, the team in Sydney. Yes, we have a team in Sydney. We are growing and we're rapidly growing across. So if you're interested reach out to us and we want to sort of build together. So now the thing you've been waiting for the winners.

[01:02:15]So, in third place, to recap, they will be receiving in addition to one year of ChatG Pro per member for an entire year, $25,000 American dollars in OpenAI API credits.

[01:02:26]And that goes to drum roll please Royce.

[01:02:38][applause] >> Congratulations.

[01:02:42]Now in second place 50 grand and that goes to drum roll please >> EI Wizards [cheering] [applause] >> and for first place they will be receiving 75 grand in OpenAI API credits and not drum roll this is not a small amount we hope that we sort of build something amazing Build the future with this. And drum rolls please. Let the whole world feel the world shaking. And that goes to >> Hamage Baltitude. [cheering] [applause] So, thank you everyone. We have pizza.

[01:03:39]We have drinks. It's going to be a great night. It's been a great day of building, great day of cooking. Thank you. See you soon. Goodbye.

[01:03:47][applause] >> Okay. Uh let's get the winners up. Uh all the the top five and let's take a photograph.

[01:04:10]Yeah, >> not even a full year. Feels like lots of fun.

[01:04:31]I was at data science and >> [music] [music] >> Oh, okay.

[01:04:52]In the US there's about 50,000.

[01:04:56][music] [music] All right everybody [music] join [music] ahead.

[01:05:32]>> [music] [music] >> Instagram.

[01:05:43][music] All right, everyone. Make sure that you can see my camera.

[01:05:49][music] [singing] Three, two, one. [music] >> All right, everyone.

[01:06:00]Yes.

[01:06:11][music]