What the new /posteffect command REALLY means for vanilla shaders

[00:00:00]Hello everyone. So, last week Minecraft added this new post effect command and basically everyone's going been going crazy about it.

[00:00:10]Uh like you can see here, they just added all these different effects and basically how this works is uh they they're just basically taking these effects from ShaderToy. That's like a website where you can upload your own shaders.

[00:00:24]And what this does is like it takes uh screen pixel coordinate and then it does a bunch of math with it and then it outputs a uh screen color for this coordinate. So, if you for example uh the middle, that's like coordinates 0.5 0.5.

[00:00:41]And then it does some math with it, some mathematical function. And then the function outputs uh like three different numbers that each represent the red, green, and blue channel.

[00:00:53]Uh and like these represent different brightnesses along the blue channel in this case.

[00:00:59]Uh and the idea is in Minecraft you can do the exact same thing.

[00:01:03]And uh just basically copy and paste this code into Minecraft.

[00:01:09]Uh and well, that's what they did and it's pretty cool because you can uh like insert all these different effects.

[00:01:18]Uh and basically what I was thinking like right right here you can see these are actually 3D effects. Uh like right here it was just uh a 2D pixel coordinate.

[00:01:32]But these are like 3D.

[00:01:35]But the principle is still the same.

[00:01:37]Like uh you can see here. This one's also a 3D effect and when you scroll scroll way down uh you can see it's still just a 2D pixel coordinate and uh well, then it just does some geometry to basically uh turn this 2D pixel coordinate into like a a point in 3D space and a direction that you look at uh like right here.

[00:02:04]Like a camera position and uh a vector uh pointing in the direction that you look at and then it casts this ray inside the world and just basically decides which color does it hit.

[00:02:18]Uh and this is how all of these different effects work like this one uh and this one. They They all look pretty nice.

[00:02:27]And basically what I was thinking uh all of this stuff is just some fixed camera positions like uh it works pretty well as a screen saver.

[00:02:37]Like for example Phoenix SC also did a video about it and all of the people in the comments are saying well this is basically a screen saver.

[00:02:46]Uh and I was thinking it would be pretty nice if this was not a screen saver but instead you could just move around this like it was a regular Minecraft world.

[00:02:57]Uh and basically this is what I did.

[00:03:03]Uh so you can see right here I'm moving it around inside my world and uh this is exactly the effect that you see here but inside Minecraft and it's not just a static screen effect. You can actually move inside it and uh like it gets included by walls and everything.

[00:03:24]You can also uh like go to the overworld and see how it looks.

[00:03:29]And it's pretty nice.

[00:03:32]Uh and now you might be wondering like I I explained a bit but how does this actually work?

[00:03:40]Uh and the basic idea is that uh this ray direction like uh when you uh look in here here.

[00:03:54]All of this code is just for setting up like which position in the world is the camera at and which uh which direction does the current pixel uh which direction in the world do you have to face to point at this pixel.

[00:04:09]Uh it does all this different math and then it's just inserts it's into this camera setup function and calls a uh render function.

[00:04:20]And then it has this color and outputs it to the screen.

[00:04:24]Uh so, I was thinking if we can go inside Minecraft and do like a bunch of math that tells us which direction each pixel on the screen points to.

[00:04:35]Uh we can basically insert it into this code and uh we have this effect working inside the world.

[00:04:44]And well, you might be wondering how does this actually work. How do you calculate this ray direction?

[00:04:53]Uh and there are two answers to this.

[00:04:57]First, it's complicated and two, it's actually quite simple.

[00:05:01]Uh so, this is basically my project setup and uh the core of the of the whole idea is um >> [clears throat] >> like all of this uh of this random code does like a bunch of matrix math to convert between between uh screen space and world space. Like um this is the core shader. Like this is what's used to draw all the things inside the world.

[00:05:32]And uh the game basically has internal logic like uh this uh leaf block is at I don't know which position position uh like negative 116 73 minus 90.

[00:05:48]And then this position inside the world needs to be converted into like which screen coordinate, like uh X and Y.

[00:05:57]Where does this block need to be drawn?

[00:06:00]Uh and this is basically the logic that is done by this core shader. Like all the core shaders are used by the game to draw the things in the world, like the blocks or in this case like item entities in the world.

[00:06:13]Uh and all of this uh conversion sounds pretty complicated, but it's actually just this one line of math.

[00:06:21]Uh you don't even need to understand how matrix multiplication works, but it's basically just um like some function that when you multiply the position by uh this weird object, you just get a screen position out.

[00:06:43]And this is basically uh the whole idea.

[00:06:46]Because uh because this is like a mathe- mathematical object, you can just call inverse on it.

[00:06:54]This is a built-in function in uh OpenGL, like um just takes this mathematical object and uh gives a new one that that's just uh when you multiply it's back, it's it just undoes the whole operation.

[00:07:13]Uh and then when you somehow output this into the into the post shader, like um uh it's been a while while since I read this, but like right here.

[00:07:28]When you output this into the post shader, uh here I have it output.

[00:07:34]Uh you can just multiply it back into the screen position.

[00:07:38]And well, then you get a world position.

[00:07:42]and this is basically the whole trick.

[00:07:45]Using this, you can take any position on the screen.

[00:07:50]Uh and multiply it with this inverse matrix, and it basically gives you a direction inside the actual world. And this is then what I can enter into the ray tracing logic.

[00:08:03]And then it just works.

[00:08:04]Uh I can plug in any shader code shader toy code uh that I want.

[00:08:10]Uh and then the next question is, this is in the core shader, and then this is the post effect shader.

[00:08:18]How do I actually transfer the data between the two?

[00:08:21]And uh I've been hovering over it uh with my mouse a bit.

[00:08:26]Here in the bottom left corner of the screen, uh maybe if I make the screen large, you can see it.

[00:08:33]Here in the bottom left corner, you can see some uh marker pixels.

[00:08:39]And actually uh when you go into the core shader, you can see it like uh this is the vertex shader. This one's uh one time for every corner of the block, and um then I'm out outputting it into the fragment shader. This This one's for every pixel of the block, and basically decides uh like which pixel which color should be drawn here.

[00:09:07]Uh so like right here, you can see the already existing game logic for looking up the uh the leaves texture.

[00:09:18]And uh what I just added here is uh like if this special entity is currently being uh handled uh in my case, I'm just um using like a command to spawn a custom item display.

[00:09:36]Uh and I'm doing a test if this is if this item display is currently being uh rendered, I'm uh setting this sent data variable to one. And then in the fragment shader, I can test for this.

[00:09:50]And uh basically just run any logic I want.

[00:09:55]And in this case, um I'm just testing for the uh for the pixel coordinates. And depending on the pixel coordinate, I'm um encoding a different part of the of this matrix.

[00:10:10]And this is basically all the logic that is behind it. And that's how I'm uh writing these uh these marker pixels.

[00:10:21]And then inside the uh post shader, I'm uh I'm outputting this uh this matrix from the vertex shader to the fragment shader.

[00:10:34]The idea behind this is basically the vertex shader only runs one time for every uh corner of the whole screen.

[00:10:41]Uh so this is basically more efficient if I read it in here.

[00:10:46]And uh yeah.

[00:10:51]Uh I forgot how I'm actually setting this. All right, using a a for loop. I'm just iterating over all pixels on the screen.

[00:11:00]Uh or like or just up to four pixels.

[00:11:05]And I'm iterating over these and uh reading from the screen and decoding it and writing it into the into the matrix.

[00:11:15]And this is basically all that is to it.

[00:11:21]It sounds pretty complicated and I admit it is, but I hope I could make it a bit understandable how this works.

[00:11:30]Uh but now comes the actual uh, the actual mind-blowing part. This is Minecraft 1.21.10.

[00:11:42]The change log that I read at the start of this video was for 26.3. So, this was like 1 year ago.

[00:11:51]And now people are freaking out about being able to, uh, render all these static images.

[00:11:59]And well, the question would be why are they freaking out if I was able to do this 1 year ago?

[00:12:06]Uh, and well, it's uh, the answer is actually pretty clear. Like, you have just these simple command blocks and you run one command and, uh, you do like half of what I did here in in all of that.

[00:12:23]Uh, and I've also seen some other people, like, um, >> [clears throat] >> uh, use these commands to send data to the, uh, post shader.

[00:12:34]And, uh, in my opinion, it's pretty inefficient.

[00:12:38]Like, they used one command to send one bit. And right here, I'm using this tiny area of the screen to send, uh, like, this whole matrix.

[00:12:50]Uh, but yeah, I I just wanted to show off what what it actually looks like uh, when you do this stuff. And I hope people can use my approach and do some cooler stuff with it because, uh, like, this is a pretty cool start. Uh, most people haven't hadn't even noticed that you can do this shader stuff.

[00:13:11]And now that, uh, Mojang added this new command, I hope, uh, people get more into it. Like, I mentioned PhoenixSC.

[00:13:20]Uh, he already showed my portal project, but he seemed pretty confused by it. And now he started to actually write his own code and like I I hope people start to share my passion because this stuff is pretty cool.

[00:13:38]Uh so this was about everything I wanted to show. I hope it wasn't too confusing.

[00:13:43]Uh I hope what I showed you had a bit of a point.

[00:13:48]But yeah, thanks for watching. Thanks for listening to me and goodbye.