Shader Update - Dynamic Scaling & Foveated rendering

[00:00:00]Hey, what is this time traveler beard and this is a mid-month video for the extra shaders. Um, I wanted to show the menus and some options for the stuff I was just been working on the last 2 weeks.

[00:00:09]As you guys can I can enjoy it.

[00:00:11]Uh, what we're on here is a configuration that is based off of medium PT.

[00:00:16]Um, this new stuff that I did is was to try to make the dynamic ray trace scaling better so that it's actually more usable. Um, I have not had it actually active by default at all in the shader for a long time because I felt like the mirrors and water were very low quality with it on.

[00:00:35]As what it's doing is, um, not ray tracing every pixel every on the screen every frame. So, things like perfect mirrors just look bad with it. But, I've done some workarounds and some fixes and I'm going to show them.

[00:00:48]Like I said, this is the PT medium. I'll also show an ultra and a um, a high. The ultra has insane FPS gains. It's running like seven times faster, probably like five to seven times faster. Uh, but what this stuff is inside of quick performance, we have our light upscaling and denoising.

[00:01:06]Um, it should be on anyway.

[00:01:09]Um, you set an FPS goal and you set the minimum quality of lighting you're willing to endure.

[00:01:15]And ideally for this to actually do well, you would, um, set this to something realistic. So, if this is getting 48 on the settings I've got, um, then I should set this to like 45 or 40 or something and then it'll it won't just stay at this lowest quality lighting, but it'll kind of move around dynamically with the FPS to give us a better picture while also boosting our frames. So, don't just cram this to like 240 and think you're going to get 240. It doesn't work that way. Um, it just lowers the the quality of ray tracing down to this to try to get to that.

[00:01:51]Um, so the reason this wasn't on before and the the way I'm getting around it is the stuff in the quality enhancers.

[00:01:58]Um, I also added some new things in noise ghosting and detail.

[00:02:01]I wouldn't mess with it much. One of them mainly is the uh the denoising mode four.

[00:02:07]And I changed the uh the sampling for multi-frame upsampling on the lighting to have a halt and a fast I guess I'm using fast halt and then a blitz.

[00:02:17]Um, I got rid of some of the old ones cuz they were buggy.

[00:02:20]But, this is the meat of this is quality enhancers.

[00:02:24]So, we've got uh foveated rendering and a bunch of other stuff in masked high fidelity pass.

[00:02:31]Uh this distance occlusion stuff I wouldn't mess with.

[00:02:34]It generally works. If you don't like things look too bright or something when you first turn the camera or if there's like ghosting behind enemies that move that's too bright, turn this off and it'll be black instead.

[00:02:45]Um, but we've got the render mode and the masked high fidelity pass.

[00:02:51]What this does is it's utilizing the second light bounce pass which we usually didn't use um to add more detail since we're doing the whole scene at lower detail with the dynamic quality.

[00:03:04]So, if we put this to two, so we have a second bounce light pass and use it as a masked high fidelity pass, that's when all this happens.

[00:03:11]Um, I do have a debug view.

[00:03:14]Um, I'll see probably ignore that for this.

[00:03:18]But, you could watch the last video that shows that. Uh the options are we have foveated rendering which keeps a segment in the middle of the screen that always renders at the highest quality.

[00:03:28]Um, so that even if you have the quality really low or dips really low for a second, where you're looking is still going to look good. This can however look buggy sometimes cuz you'll have like you'll see the circle. So, I'm going to work on the fade on that and the shaping of it. You can adjust its size which greatly affects performance cuz then it's doing the full tracing on more of the screen.

[00:03:49]And how much of it is used for fade.

[00:03:51]Um this will by default automatically support any VR mods that expose eye tracking to it, which I think would be excellent. Um as it says here, they would just need to expose uniform vec2 eye tracking, and the shader will automatically utilize that for the foveated rendering.

[00:04:11]I'm excited to see if somebody does that, cuz I think it'd be really cool. I never really thought of Icarus as a VR shader, but there's at least one person who really liked it for that, and uh they might appreciate this if that ever gets added to Vivecraft or anything else.

[00:04:23]Um just occlusion tracing. This is another option to make things look better when they're first on screen.

[00:04:28]This can be very heavy.

[00:04:30]Um I thought I had a warning on there to say it's very heavy. I probably the frames is what it warns is very heavy. Uh what it does is it guarantees a certain amount of tracing when things are first on screen, so they don't start off at a very, very, very low quality, but have some data at least. Um boosting this is not a good idea. It absolutely destroys frames.

[00:04:53]Um what I want to do is change the resolution of this high fidelity pass, so that it can be at a lower resolution, because right now with it being full res, when you look at mirrors or water or something other stuff that that uses this, um it tanks your FPS basically to what it would have been before, because it's doing a full res trace on every pixel.

[00:05:13]Um and that's not that's not efficient. Um if we did quarter res, it would probably be fine.

[00:05:20]Um so looking at the actual game again, here we are in the scene.

[00:05:26]Um so you'll notice, let's look at the floor for example, uh when it's first on screen, the detail is low, then you see that detail come in.

[00:05:35]You can also see the foveated rendering spot there, where I really want to uh kind of adjust how that works, cuz it's a bit too obvious right now, but you'll see like it keeps that super high detail where you're looking, even as soon as you look.

[00:05:49]Um And that option isn't on by default.

[00:05:52]This is all just in these configurations that are included for now. So, if we switch to render mode by hiding the GUI, you'll see everything immediately goes to a higher quality and it starts accumulating uh much faster and gets that detail on the scene uh really quick, regardless of what was going on before for our quality settings. Try to keep the FPS high.

[00:06:17]Um So, >> [sighs and gasps] >> yeah, you don't need to use the foveated rendering or any of that if you don't like it. Um it's all just there to as an option to uh make things look better where you're looking. Um So, let's look at the some of the other ones. I'm going to switch this to ultra into high configuration.

[00:06:39]Um and also show you what the differences are between them.

[00:06:43]Um I should just [music] put them right here. If we switch off of this temporal checker boarded mask for the light simulation quality, we just go to quality mode, it does twice as much tracing every frame, um which is why I I made those is it just made it the shader twice as fast usually.

[00:07:02]Um um but it doesn't really on its own it doesn't really do much except that it's doing more ray tracing every frame.

[00:07:10]>> [laughter] >> It generally makes PBR materials look a bit better uh when you're on this dynamic scaling going down this low.

[00:07:18]Um it's that makes it possible to then also do things like put this to close it with ultra. The ultra would have one other setting before that I have not revamped yet. So, I'm not testing that right now. That other setting also will not work with photonics. I am I am using photonics on these presets.

[00:07:37]Um I do I have found that the photonics RT API is faster than my DDA trace and the little bit of acceleration I had to do on it.

[00:07:47]So, I would recommend using the Photonic Tracing and you need to download the Photonic mod for that to work.

[00:07:53]I'm using version 0.3.4.

[00:07:57]Um I've not gotten version 0.4 working yet.

[00:08:02]So, here we are pretty much on ultra settings and we are getting 20 something FPS where is that would normally get five.

[00:08:09]Um So, here we're going to especially look at these sun glints where these glints are coming off of sunlight bouncing off of these metal things onto other areas of the screen. If you look on the ceiling there when that stuff's first on screen, it's all blurry.

[00:08:22]Then you see it gets resolution over a few frames. That's what the dynamic ray trace scaling is doing.

[00:08:28]Um and you can see it's hooked into the denoising to make it so instead of being noisy, it's just lower quality. It's kind of blurred out and then it gathers that data over a few frames. I think it's doing a very good job of of what it's doing to kind of to pull this off.

[00:08:44]Um Because this was running at five FPS these settings before. Now, I can generally run around at 20 to 30 on the the ultra settings. Um And the foveated rendering is like we see like where if I look, it's automatically higher quality.

[00:09:01]And then anything on the edge of the screen just takes a few frames to catch up. So, that's pretty much what the dynamic ray trace scaling does.

[00:09:09]Um that's what the foveated rendering does.

[00:09:12]And the disocclusion stuff um like I said, guarantees that when you turn the camera fast that that first image on the screen Oh, no. Um isn't going to be absolute trash, but it's going to have some data there to start with.

[00:09:27]So, um before I start turning things off and showing how bad you can make this how broken you can make it. Um I just want to say thank you to my supporters on Patreon and YouTube. I forgot to thank you guys in the video for the month. So, thank you very much. Um, here's the VIP members on Patreon and YouTube. Uh, thank you everybody. All of your support keeps me going on these projects. Um, it's been an honor to um, be able to explore this hobby and something I wanted to do and learn about with graphics programming and shader dev and um, be able to share with you guys both as my journey of learning and showing it and I'm doing the tutorials and all that other stuff and making all the other shaders that that run better and then this one that's trying to push the limits. So, thank you so much for uh, for joining me on this journey and helping me go forward with it.

[00:10:20]Um, Okay, so the fluffy dragon, you notice it real bad here for some reason uh, because um, when we're doing a lower quality trace the the reflections and stuff especially can lag behind when it's not tracing every frame.

[00:10:36]Um, if we switch into that render mode, of course, everything looks better.

[00:10:40]But the the whole idea of all of this is to make it uh, more playable on different settings.

[00:10:47]Um, these settings would probably never be playable, but if we go into the render mode, this is probably what it is would run at at full screen tracing, which you can see is 5 FPS.

[00:10:59]Um, and then when we get off of that we're running at 20 22, 31. So, it's it's doing a pretty major boost.

[00:11:10]I expect that for most people um, you would want to use more of the the medium or the high configuration with this sun quality um, but then not doing the sun glints.

[00:11:24]I feel like that is kind of a better move.

[00:11:28]Um, I'm not going to put it back on the the medium but putting this down to the DRS and portal because it compiles incredibly slow on my machine. Nvidia drivers don't seem to like what I'm doing with that and they just take forever sometimes.

[00:11:42]So, on these configurations, I am not using this trace distance. I'm using protonix bounces because um on the main menu, if we were to go to if we were to testing, protonix, I've got this sound to use its API.

[00:11:58]Um So, let me show what turning off the foveated rendering and some other stuff looks like and why that exists. If I turn off the disocclusion um stuff in here, um we'll get more FPS, but the when stuff first comes on screen, you might see it look kind of dark or off or more noisy.

[00:12:21]Um so, you'll see why that exists.

[00:12:24]I wanted to switch this over to compute passes. The last video was about this uh cuz then I could do more like erratic tracing instead of having stuff be grouped together.

[00:12:35]This is so all over the place on the screen that it wastes a lot of um processing. Um some pixels do it and some don't, but they all run together in groups.

[00:12:44]So, if I can get the compute shader to not suck on the FPS, this will be a lot faster and I could do some other really cool stuff I want to do to make the foveated rendering look better.

[00:12:53]Um but with the disocclusion off, cuz you know, when I turn the screen, there's more of a um it just off and it's dark for a second.

[00:13:02]So, that's what that disocclusion tracing fixes.

[00:13:10]And why it's in the quality enhancers to let us put the rest of the quality really low.

[00:13:13]Um I'll put that back on and I'll turn off the foveated rendering to show what that does.

[00:13:20]Basically, if this is off and this is off and this is off, but there's pretty much no reason to use the mouse while the lady pauses. That's what it is doing.

[00:13:29]Um So, here we are without the foveated rendering.

[00:13:33]Um So, we see we no longer get that kind of bugginess of that circle where there's you could tell the edge of it, which I I do want to work on.

[00:13:43]Um but everything is kind of kind of laggy.

[00:13:46]And let me see if I can get back to the normal map testing over here.

[00:13:49]Um you'll notice that we don't always have high quality in the middle of the screen.

[00:13:54]So, everything takes a second. If I go to render mode, let's see, pick up, and then go back down.

[00:13:59]Um we see especially these birch trees, for some reason the Patrick's textures birch trees and this technique do not get along.

[00:14:08]Um But if I had the foveated rendering, it anything in the middle of the screen would always be high quality and it wouldn't have that that bugginess going on.

[00:14:17]And you can see how like where the texture takes a second to fade.

[00:14:19]Um That wouldn't be a problem. So, let's let's push this even further.

[00:14:25]Um I'm going to go back in here.

[00:14:28]We'll put back on the foveated rendering.

[00:14:30]And um let's look at the difference of this minimum quality. Like I said before, this is always pegged at one.

[00:14:37]Um if you want it things to go faster, doing something like this um probably keeps it to like a 3x3 tile of what's being traced with one pixel every frame.

[00:14:51]So, that should probably be very responsive. And you're going to have diminishing returns at some point. If me putting this to 0.5 gets me 52 FPS, but putting it 0.11 gets me 50, I'd rather have it at 0.1 cuz the quality is going to be so much better um than the than that little bit of FPS.

[00:15:14]You see this loses us a few frames, but it's going to guarantee you that the quality of everything being traced is always higher.

[00:15:21]So, you can see it takes less time for areas of the screen to resolve and they resolve to a higher quality um even when we when it doesn't can't meet its FPS goals.

[00:15:33]Um since we are hitting 40 FPS I might Well, put this super low and then I'll make this stuff work as intended.

[00:15:42]So, putting it higher guarantees us better quality.

[00:15:46]Um and I'll put it really low. This is only tracing like one pixel per line on the screen probably per frame. This is one out of every 2,000 pixels getting traced per frame.

[00:15:57]But again, we will probably see diminishing returns.

[00:16:00]We're um the rest of the shader is going to keep us from getting to a point that this is worth it.

[00:16:07]So, um you can see especially when the shader first loads, um it doesn't have things flagged right necessarily. So, here this is the quality we're getting with that. So, we're getting 50 FPS.

[00:16:16]Uh you can see when things are first on screen, they don't look broken. They don't look anything. They just kind of kind of low quality.

[00:16:24]Um but wherever we're looking is still super high quality. Especially with VR, I think this would be extremely effective.

[00:16:30]Um but it's passable. Um you And if I go into render mode, you can see everything catches up.

[00:16:38]Um but there's definitely you can notice very much where the foveated rendering is and is not because especially with specular reflections, um if we're only updating one out of every 2,000 pixels, we're going to have a difference with uh reflective surfaces.

[00:16:56]So, definitely diminishing returns. But since this gets us to 50 FPS, let's look at even if we had this um See, 45. So, we've got some room.

[00:17:10]Um this should then now go down to that but not stay that low and boost itself up to higher quality to try to keep this frame rate. I don't think this is perfect.

[00:17:23]So, this number might not actually be what it actually in practice even out to. Um but we'll find out.

[00:17:32]And there's debug view here.

[00:17:34]Um I don't know.

[00:17:38]Oh, that debug view apparently does not work. I'm going to have to redo it. Um it looks like it broke with my shader parser.

[00:17:48]Um so, if it's to something that I can get, then the the quality should fluctuate.

[00:17:54]Hopefully do somewhat better. But you see it being um so low is not going to be great.

[00:18:00]But if you're just trying to run around and have quick responsiveness and then take screenshots, um putting it super low might be good to kind of get in place and then switch to like render mode there. And again, render mode isn't going to change any other options. It just makes it trace every pixel fully every frame.

[00:18:17]Um You see with a very low how bad uh subsurface scattering suffers. Um And then if we go to render mode, it does better. Cuz it's just that's not enough information. But if we um are running at um the ideal use of this would be let's say if we're currently getting around that. Let's say we want to get 40 and we're willing to go down to to this level of quality or maybe one out of every 20 frames or whatever.

[00:18:51]And if you aren't using like a realistic resource pack with good normal maps, you can probably get away with this being very low.

[00:18:57]Um But even on this, like this should do better. You see it it the edges of the screen don't look horrible.

[00:19:07]Um even though this is only tracing one out of every 20 pixels on the screen, um it's still resolving at a fairly good amount of time.

[00:19:15]And, um, the foveated rendering is kind of obvious.

[00:19:19]But, um, if we're keeping the uh, minimum quality high enough, um, we could probably turn it off and be fine without it.

[00:19:33]To tracing, um, at point zero five quality, um, here we are.

[00:19:47]And see, it's good and responsive for like edge of the screen and everything.

[00:19:51]Um, and this is not doing the foveated rendering, so we don't have that spot in the middle. Um, but instead there's any sort of specular stuff is going to have a little bit of a delay in negotiating uh, just because it's has to accumulate that data.

[00:20:06]Um, but there will be, um, more options to solve that better later. And let's look at if I put this at something, um, a little bit higher and probably more practical, like like even just like point one.

[00:20:22]Something like this probably wouldn't need the foveated rendering and also would look the best with it because it wouldn't be really obvious.

[00:20:29]Um, and with that even if I switch to render mode, um, aside from like the birch, which I hate this, see everything else pretty much already looks the same.

[00:20:41]Um, so it does a very good job in in that sort of a case of And like I said, there's diminishing returns where, um, it's going to look good, um, but then at some point it's going to, um, just be losing quality and not getting any more FPS.

[00:21:03]So, that's what you'd want to try to be aware of with tuning it.

[00:21:07]And the little flashes on the edge of the screen, that's where the um the seclusion cache thing is is having issues there.

[00:21:15]So, here we are standing here and there's render standing here render. I'll put the um foveated back on.

[00:21:24]Show that some more.

[00:21:26]And then I I I do I need to show the mirrors because I was um talking about that. I'll actually show with the debug view through the mirrors.

[00:21:36]So, let's get over to these metal area.

[00:21:42]And that the disocclusion tracing does not do well in outdoor scenes um with lots of grass and stuff.

[00:21:49]So, let's look at um mirrors here.

[00:21:57]If I go in here and override metals to be mirrors.

[00:22:04]Here's since we are using the protonics API, oops.

[00:22:09]I went to the I know you.

[00:22:13]Um we have the um glass and everything is it's voxelated version and we got the really good off-screen detail on things.

[00:22:23]Um And you see the mirrors have full res tracing. But again, I want to kind of lower that to be um less of an FPS hog by doing it at like quarter res instead.

[00:22:37]And if we switch this back over to what would be pretty much ultra, but turning this on here, then we should get back um these caustics, reflections off of the stuff, off of the metal.

[00:22:50]Um and doing this is probably a spot where having the lower quality is definitely worth it.

[00:22:58]Um I think I had it at this before.

[00:23:01]And I'll show the difference of in my photonics my use of photonics right now, it does not detect water. So, we miss the water caustics bouncing off of them.

[00:23:10]Um that's something I can fix later and just have not gotten to. I'll show it with that off as well.

[00:23:18]Um there we go. Um and you could see from the FPS that um using the photonics trace apparently helps a lot.

[00:23:26]Um it plays very well with my dynamic scaling.

[00:23:29]Um versus my tracing is it looks like the photonics is twice as fast at least.

[00:23:35]So, I definitely recommend using the photonics option and using the photonics mod. So, that's in photonics you just turn this on. Everything else should be set how it should be.

[00:23:48]And um again using all this masked extra pass um >> [snorts] >> hurts the FPS, but it it it adds back the quality that you're you're losing when you gain all the all the FPS from using the dynamic ray trace scaling.

[00:24:09]Um but I definitely want to do more with it to cuz I love getting stuff like these glints that like like a frame rate that's not five FPS.

[00:24:19]Um but um there definitely is room to do more about making the foveated rendering circle not as obvious.

[00:24:30]Um and doing other things like making the disocclusion tracing not destroy FPS as bad.

[00:24:38]Um that's all there is for this video though. I hope you all enjoyed the video and thank everybody for your support and your interest.

[00:24:46]And um I'll see what else I can get done before the end of the month.

[00:24:49]Um it'll probably be some work on this and I want to do some stuff with froxels to make the fog not suck.

[00:24:54]And um I do want to get uh Photonic's version 0.4 working.

[00:25:02]Uh because with that, I would get um per pixel emissives in the uh the way things glow.

[00:25:11]So that um torches would glow just from them, lanterns would glow not from the top and bottom, just from the sides.

[00:25:18]Um stuff like that.

[00:25:20]Um and pumpkins and stuff would only glow out their holes, not like the whole block.

[00:25:25]So um it would at least make some good screenshots.

[00:25:29]So thank you again. I hope everybody enjoys it and I hope that you get some enjoyment um and some more use out of this.

[00:25:36]Uh if you were playing on medium, um just have it on medium but with these, and then you can always just get some screenshots real quick too with the screenshot mode. Have a good week. I'll talk to y'all soon.