Virtual production technology enables real-time integration of physical cameras and objects into digital environments by using tracking systems like Axymmetry to capture camera movement data and transmit it to Unreal Engine, allowing physical spaces to be recreated and manipulated in virtual worlds with features such as destructible walls, interactive avatars, and real-time camera tracking for immersive streaming experiences.
Deep Dive
Prerequisite Knowledge
- No data available.
Where to go next
- No data available.
Deep Dive
I'm Upgrading My Studio To The Next Level with Unreal Engine | Office DrummerAdded:
But yeah, this is kind of what we got going on in Unreal right now. Um, we'll also be able to give you guys custom animations. We could give you guys pets.
What would What What do you guys want um uh to be able to do after we spawn? What will the avatars be able to do? So that's totally up to us and uh where we want to take this cuz you could do so much.
Definitely dance. Definitely mosh.
Show them my fox's spot. Okay. Yeah. So we will have a secret spot because Pete Fox has been so helpful that there is going to be a secret spot for the fox up here up here in the venue. So, just sleep it in the corner over there somewhere. You know, we can add like chances uh like if you gift a sub, there's like a chance that you'll get a rare item or something. We can do things like that.
Basically, anything that you can do in a video game will be able to do.
Man, it's been a crazy crazy few days, man. I've been working so hard, dude.
Um to get this this going. Uh, I'm so exhausted, but I'll be okay. We're not only using Unreal, but we're using another program called Axy Symmetry.
And what Axymmetry does is allows you to bring in um actual cameras. And then you can use tracking information from the camera like lidar or whatever to send that to aximemetry which then sends it to Unreal. So if you move the camera like this um your background in Unreal will also track or however you move um will also track in Unreal. So, it'll start to look really realistic and immersive so that we don't just have static cameras and uh digital movements.
We can have physical cameras moving as if they're in Unreal essentially. Well, I'll give you guys a little lay of the land right now so you can you can see what we're working with. We got the drums over there. Um things are kind of not set up exactly right currently, but hey, that's what we're going to work on today. You know what I mean? We're setting up a virtual production studio.
So, I have the green screen on the floor. I cut out little bits for um where some of the the the things that like to slide around on drum sets go. Um but yeah, we've built this whole uh round uh mounting system with seastands and everything mounting lights. We have another pole here for another crosssection of another lights.
Um, but yeah, my drums used to be there.
They still are there. Same room, just new skin, you know. So, I'm going to have uh an artist recreate the studio the way it was, exactly the way it was.
And um so you guys will will still be able to visit it uh studio 1.0 but now we can make that environment interactive.
So the legend of the month all the stuff that was on the wall now you guys will be able to directly interact with it.
And I don't want to spoil too much, but uh what I want to do is make the walls destructible.
So, studio 1.0, but with destructible walls. And then we can place that studio into other maps so that as you guys destroy the walls, it'll reveal the the new map that we're on. So, we can put it in the middle of the coliseum. We can put it on the moon. We can put it underwater. We can put it on on a aircraft. Like anything.
So, uh like I said, what the creative juices just start flowing, man. And they just do not go away, dude. We got this. We printed this out. Our uh whatever dot And we will use that. And then we have our phone which is going to send the uh tracking information to the computer through an Ethernet port.
Look at this beast of a thing that I've been working on, man.
So, okay, it's a little hard to see, but here's the track.
Now, this thing runs along it here.
Actually, I can show you guys with the uses a PlayStation controller.
Okay, let me set the correct mode on here.
Oh, is it working? Yeah. So, look at this.
it moves and this can be set so that uh eventually you guys should be able to control this as well. So we have multiple points of uh of movement and that's all tracking information that we can send to a symmetry uh which is cool. So I can move the gimbal point you obviously the camera is not on it right now but um we will be getting the camera but yeah this thing can move and you can set it to move automatically. I'm just doing it manually right now but uh this will capture a shot that goes from all the way over here to all the way over here. And we can pan it. And then we can also extend these tracks out and have this thing basically move over me. Oh, what's going on?
Hey. Hey. Hey. Whoa, buddy. That's never happened before.
Now you guys can see why I sp I've been spending 12 hours here every day. It's like, "Oh, well, that's a brand new problem. Haven't had that problem before."
This gimbal in particular has been um a huge learning curve in the setup to all the tech involved with it.
But we love it. We love it, don't we?
This is the lens. I don't know if you guys can see this. Oh, you can. So this is the lens that's going to be attached to this. So this is the lens that we need to calibrate for tracking right now.
So when we're doing an IRL stream, we can actually have this computer on running Axy Symmetry, running um Unreal and we can bring in elements of that into IRL streams.
So you guys could spawn in into an IRL stream. Um, as long as we have a tracking device on our um on our IRL camera, it'll send that tracking data to Unreal. So the AR possibilities are actually really huge, too. Uh, so very exciting stuff.
You see this thing? OH MY GOSH, IT'S FALLING. AH, I've been spending like five hours a day on this freaking ladder, dude.
Of the 12 hours I've been here, I spend like half of it on the ladder plugging things in. I'm actually really looking forward to the cable management of this, especially with uh an underlung overhead dolly gimbal setup. The cable the cable situation has been out of control, dude.
Like the cable situation is literally a nightmare, man. What cameras are you running? Blackmagic? Yes, these are Blackmagic uh Micro Studio. Previously, I was using Sony A6000s or A5000s, which are HDMI, but these ones are SDI. This is my first time really working with SDI cables and cameras and stuff like that. So, okay, that's good enough for me.
Next, the origin is not set yet. You have to use the detect button or use a different method.
Make sure Oh, detect. Oh, I'm in the way.
Uh oh. This might be an issue actually.
All right, let me get out of the way.
Detect.
Origin found. Let's go, baby.
Place a distinct vibrant red flat object in front of the camera.
I guess that has to be MY CROCS, RIGHT?
NO. GOD.
NO. GOD, PLEASE. NO. NO.
IS THAT FLAT AND RED?
Is that good enough?
>> Whoa, look at that.
>> Marco number one is where it's supposed to be. It's it's it's in the uh middle of that code, right? That's why I told you that it has to be in the same place every time. Now, marker number two, turn that one on. And do not press any of those put in front of camera buttons yet.
>> Okay. I was very tempted to press those.
So, thank you.
>> Yeah, I saw that and I was going crazy.
[laughter] Now, what you want to do over um uh you can you can press put in front of this camera on the camera under marker two.
Press that one.
>> Put in front of camera.
>> Yeah. But only Yeah, that one. Yeah.
>> Yeah.
>> Then you should see it pop up somewhere.
Does it do that?
>> I don't see >> too. Okay. So, what you can do now, try and and and move the camera around a little bit.
>> And then we should see you should see the the the grid like stay where it is.
>> Yes.
>> Move it to Oh my gosh, look at that.
>> Now, now what we want to do here, you can see Oh, you can see marker two down there in the back. Try and and tilt the camera up a little bit.
Now, what you want to do with marker two, you want to move that so that it sits where your drum set is.
uh and you can do that by changing the position numbers uh over there by micro two. Now the reason for that is that when we have multiple cameras then we can pretty easily make sure that everything is where it's supposed to be um and and and recalibrate things because you're going to have to do this every time you start.
And that's why I told you you want that QR code in the same place every time.
>> Oh, really? Every time.
>> Yeah, but but but not not the whole thing. You just do a one, you know, click calibration thing. It they just need every time you start, they need to see the QR code so they know where the camera currently is in the room.
>> Oh, okay.
>> And then you just calibrate.
>> But you want you want to be able to check that everything is right by first seeing marker one in the middle of the QR code and seeing marker two in some other place where you know things are supposed to be every time and that is your drum set or wherever you sit or you know maybe it's your chair or something like that. And you do that by changing those numbers u under marker 2. You see how it says 0 0 and then 2.36. If you change that up and down, you'll see the marker move in the room. Now we can move it over to where your drum kit is.
>> One hour later.
>> Oh yeah, you might want to close that one. You don't need that right now. That might be hogging the outputs.
Try and try and go into uh the other one that you were before where you selected uh the input devices.
>> Yeah. two hours later.
>> And this is this is the hard part because now we need to start measuring how high up is it and how far is it from is the phone from the QR code and a bunch of [ __ ] like that. Um, and we need to do that for each camera and we need to do that also for the for the static cameras uh stationary.
70 years later.
>> If you are subscribed to this channel right now, go ahead and type exclamation point spawn.
So, the way we have it now Oh, hello.
Hey, bro. You're in the way. Hello.
Um, the way we have it right now is like, let's say I was playing on the stage.
Uh we just have you guys up here next to me because if the camera's pointed at me and then uh shoot you shoot over here.
But um but yeah, this is like the most basic version of this, right? Because what I what I'm aiming for uh well, first of all, we'll get your guys' names on everybody.
We can save characters. So, if you guys What we want to do is make it so y'all can make custom avatars, have pets, blah blah blah. You Yeah, you can change your character. Exactly.
Custom characters, customizable characters, uh different things like that. Um as far as And that that's just one aspect. I'm gonna be bald chief. Oh my god.
[laughter] And like other things that we can do at least right now in like the most basic of setup is uh is like for bits. We can spawn things. I mean we can do anything for anything.
You know what I mean? Like you could set someone to spawn just by anyone typing in the chat. But the way that I want to do it or I have it set up right now is like for bits like it'll drop bananas on top, right? Just generates a bunch of banana and you can just generate a bunch of bananas. You know what I mean?
Like you know what I mean? You can make it rain with bananas. And the bananas can be anything, right? It doesn't have to be bananas. It could be anything. Like wigs. Oh, what? Where's What? Oh, wigs.
wigs can spawn and also it's going to run smoother and look better than than it is right now.
Um, it's not in its most ideal like state or whatever. So, it's not going to be like lagging and stuff like this, but there there is like an upper limit to the amount of stuff that we can spawn.
Yeah, you guys COULD SPAWN CROCS. OH MY GOSH, we could literally do that right now. And then also, if I want to, I can I can uh delete all you guys if I feel like it. You know what I mean? I can just be like, "Hey guys, that was a great party. Thanks for coming out.
Bye." You know, I have that power. If we wanted to spawn crocs first, that we would do is go into this and search for Crocs.
Wait, they're taking this too literal. Crocs shoe.
Wait, wait. There's no Crocs on the Fab store.
I don't believe it. Maybe they just can't call it Crocs.
Yo, Chief, this looks cool. Do I need to download anything to participate?
Yo, Bodan, type exclamation point spawn in the chat right now. Boddan, do me a favor and type exclamation point spawn in the chat right now. Call it ugly shoes. Look for ugly shoes. Wow, you guys are like, what's up with that? You guys are like one mind. If you guys want to try it again, anyone that's subscribed, type exclamation point spawn and you will spawn in. This is the the beta testing, right? That this is like the the worst version of this that you're ever going to see type thing. You know what I mean? Because first of all, you guys don't get to really customize your characters.
Uh you guys don't really have a lot of further control yet of your character, but you will. trust me. But isn't that pretty neat? Don't you think that's pretty sick? So, you do have to be subscribed though to be able to spawn.
But if you gift a sub or uh whatever, you will also spawn and or if you raid, you'll spawn in. You need to win a Fortnite match to do meet and greet in the game. [laughter] Wait, wait, that's crazy. Can we do like an Animal Crossing thing where like people from the Fortnite world can take like a a a train to us and like visit us? Like they're playing the game but then they come visit us. How do we do that? How do we make it rain gizzies? Oh, gizzy rain.
Wait, I'm going to write that down. That that that goes down in the to-do list, dude. Rain gizzies. We could make that happen right now, actually. All right.
So, we could So, they didn't have any Crocs.
Well, first of all, I'm sure they get Gizzy.
No results for Gizzy. There's got to be a free hot dog, right, guys? This makes you think of Gizzies, right?
Oh my god. Meatsuit Mommy, we can make you a Oh my god. Meatsuit Mommy can have her own skin in the game, dude. And it's a meat suit. Whoa. That's so cool.
Downloading a hot dog.
That's a weird sentence. I think it got added to the project, right?
Yeah. Hot dog, baby. Why can't I see the Gizzy?
Where is my Gizzy? Oh, at Charlie and Getty, too. Absolutely. They can follow me around, dude. Honestly, I think more like this kind of stuff is like the kind of development streams I could see myself doing in the future where like we as a community make the redeems and stuff together and like feed off each other's energies and ideas and stuff and like what you guys like and you know cuz I'm going to be spending a lot of time uh doing that. [laughter] All right, I'll see if the hot dog loads now. No. Is it just really small? You know what's cool? Actually, I bet that we could automate some of this to where we have like a link where you guys can upload photos and then those photos can be displayed on your character like like a t-shirt design, you know, used as like the mesh of your shirt. It might not be perfect, but I bet we could tweak that. Low poly dog. [laughter] That's a hot dog.
Okay. A a free hot dog bun.
Okay. A hot dog sands the gizzy. Guys, I think we're inside the gizzy.
Oh no.
This is the most massive Gizzy of all time. I can't zoom out fast enough. Oh no. It's like unveiling a whole planet.
Dude, the scaling sometimes is a little uh freaky, man. Uh, there we go. Okay. Why is that still so massive? So, that's what was happening with the Gizzy inside the studio is that it was just too big.
We should be able to bring in the Gizzy.
Oh, it's still so big. Why is the Gizzy so huge? That is a much more manageable Gizzy, don't you think? Oh my god, look at that thing. He's going to end up deleting the stage by mistake. I've literally already done so much stuff like that. I deleted the floor.
It's so because you can't lock things, which seems crazy. You should be able to lock things.
All right, but it's still way too big, right? Maybe there should be a Gizzy cannon. Dude, you know what? I bet we could give you guys gizzy cannons.
I'm just trying to spawn in gizzies, guys.
I don't know, guys. I don't know why it's so big.
What do you do? You guys want mustard?
Oh, this one looks really good.
I mean, this one's so classic, right?
Exactly. Sarasic. What do you guys like better? That one or this one?
All right. Since we paid for this one, it should just work when we put it in, right?
Whoa, that crashed my thing. See, guys, this is why everything has taken me so long. It's like, "All right, let's really quickly make a Gizzy uh command."
And now it's like what 30 minutes later and we're still >> 3,000 years later.
>> Beautiful. Look at that. Love that. And that's a goodized dog, too. Yes. Okay, this hot dog is working correctly. These are all the commands that have been built for the stream so far.
Um, let's go to bits.
No spawn wig works. So, let's try spawn glizzy.
Beautiful. Compile. Beautiful.
All right, let's give it a shot. If it works first try, chat, I'm a genius.
Yes. Look at that. It worked first try.
All of our wonderful gizzies.
How many gizzies can we spawn until my computer crashes?
It's starting to slow down. So, that's just a taste of uh what we can do in Unreal Engine and how it's going to be interactive uh for all of you guys. How did that make you guys feel? Are you guys excited about that? Because I'm so freaking stoked about that. I think it's so cool.
I think it's so cool because we can just expand on it and really um do whatever we want with it. I'm excited like nepen these holdini hold nippines hold what mods I think we have mods I think we have a first time chatter here that is being very naughty why is this $1,500 looks Pretty decent. Oh, actually, actually, let's get the floor out a little bit more.
Something like that.
>> Yeah.
>> And then we can do the green left.
>> Oh, look at that. Yeah, that's a pretty clean >> stop subs around there, right?
>> So, if you guys are just tuning in, you don't know what the heck we're doing.
This is so that we can use real world cameras that are moving um to manipulate the virtual space and the camera has to know where it is in space.
So you try to give it as much information as possible.
Okay, this is good enough for now. Now let's try and get >> There it is.
Okay. See, now now we're getting like to the point where what we want to do. Got to get the latency right. We got to get the key right. We got to get the placement right. But the idea of the drum set and you can go closer to it.
You can go around it and all that stuff.
Can wait can we work on the the room just a little?
>> Yeah. Align it just a little bit more. I was just so curious because we had it so perfect before we'll iron them and do all that before.
Uh or you know what software is this? It's called Axymmetry. So it works with Unreal Engine to um basically port in like real people, physical objects, things like that. and um put them into virtual worlds, you know.
So, it's kind of like a companion tool to Unreal Engine in a lot of ways. And it's very similar.
Oh my gosh.
Oh my gosh. Pete, look at that. My drum set is there.
If only I didn't have all the other miscellaneous crap on the floor.
>> Yeah.
>> And get that.
>> But you you'd essentially be be in there right now. And once we get the trackers proper properly calibrated and the latency and the keying done right, >> Chad, look at that. That dude, look at that guitar. That's crazy.
>> Is there >> That is so weird. like watching it and like walking up like dude once it's oh my god man >> dude with the with the nice lens the correct lighting you walk up it's going to be pe it's like it's going to take people there's going to be people who watch the stream like oh I've been watching the stream for an hour I didn't even realize that it was fake >> except for like >> dude I mean like come on like [laughter] are you messing with any more functions right now is Is it getting better or is it am I just like getting >> try try and walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk walk f f f f f f f f f f f f f f f f f f f f f f f f f f f f f f f f f f f f f f f further away from the junkit.
>> Well I don't really have that much space. There's a lot of [ __ ] back >> Yeah.
>> Um I mean that for like being the literally lowest quality version of this that we can muster up is impressive as hell.
>> Working.
>> Hell yeah. Hey, W's in the chat for Pete. Let's go, baby.
>> Pete, thank you so much, man.
>> Thank you for everything, man. Your guidance and everything, dude.
>> Yeah, no problem, man. See you uh see you Wednesday.
>> Okay, sounds good.
>> All right, bye-bye.
>> Bye.
Okay, guys, for for a final test, we will now together experience it in its halfbaked full glory. So like I said earlier, right, this is the first test of us with tracking motion in our digital virtual space. So this is the lowest tier of equipment that we have. So I only say that humble brag. No, I only say that as to say that this is the lowest quality version of this that we will ever get.
Our lights aren't set up correctly.
I woke up on the wrong side of the bed.
Enough excuses.
I mean, and this is this is this is like the bad version. Like the really bad version. Like th like this is the the crap version, dude. And look how crazy.
Like obviously we have some issue over there. You know, the lighting is not even close to being right.
I mean, but for the first opening it up and making it happen, that's pretty fire, dude. Appreciate you guys joining me today along for the ride. And hopefully next time you see this room, it'll be amazing and I'll be like, "Oh my god, this is so fun." That that's just all I want from it is people to be like, "Dang, this is fun." You know, that's all that's all it should be.
That's all it has to be. Just fun, you know? Let's make it fun. Let's make it as fun as possible.
Um, but yeah, don't worry. I'm gonna get my rest. I'm just so excited. That's the thing. It's not a drain. It's it it's uh feeding me. It's giving me energy and it's giving me energy to put back into it. So, don't worry. I'm gonna be okay.
I have a few days of rest coming up because it's Mitsumi's birthday on Wednesday and then we're going to do some things with friends. So, um don't worry about me. I'm good. Uh but thank you as well.
and I will see you guys soon. Um, I'm trying to keep up with everything like social media wise, you know, letting people know about things, but actually I haven't done anything. So, kind of just been keeping it all close to the chest, I guess, and um, just developing stuff, you know.
But, knock on wood, this could be a this could be a great year for us as a community. And, um, I'm excited. I'm really excited about it. I think it's gonna be really really fun and it's gonna maybe inspire some other uh people to want to try it. So, I have a feeling uh you know, we might get some cool collabs or something. Or like imagine the collab potential, too. Just bring a DJ in here or something and then put us on a disco moon or whatever. I don't know. Thank you guys.
I will see you soon.
Maybe for a rock talk. maybe for a launch.
Thank you for your patience.
Thank you for understanding and thank you for being here.
I love you guys and I'll see you soon.
Related Videos
Agentforce NOW AMA: Build with React and Salesforce Multi-Framework
SalesforceDevs
490 views•2026-05-28
How agent o11y differs from traditional o11y — Phil Hetzel, Braintrust
aiDotEngineer
450 views•2026-05-28
WEB TECHNOLOGIES UNIT-2 | Degree 4th sem BCOM Computers web technologies unit-2 full explanation💯✅
LearnwithSahera
1K views•2026-05-29
More tests are always better? How to use AI to identify tests that bring little value
Alliance4Qualification
335 views•2026-05-29
Search Algorithms Explained in 60 Seconds! 🤖💨
samarthtuliofficial
218 views•2026-06-01
People of Game of Thrones using JavaScript DOM
AltCampus
296 views•2026-05-30
Introduction to Problem Solving Part - 1 | Lecture 1 | Intermediate DSA
ascensionix
107 views•2026-05-29
🚀 BCS613C Compiler Design | Module 1 to 5 Schema Evaluation 🔥 | VTU 6th Sem 💯 #VTU #bcs613c #exam
Pranavaa-y4y
104 views•2026-06-02











