I Replaced RGB with Actual Physics

Added: 2026-05-24

[00:00:03]This rainbow is not a texture. Most renderers fake color with RGB. Blender can make beautiful glass, but it doesn't really trace [music] light like this.

[00:00:11]Real light has wavelengths, and each wavelength bends by a different amount.

[00:00:14]So, I built a spectral [music] path tracer from scratch, and at first it looked pretty bad.

[00:00:19]Let's start by creating a ray of light.

[00:00:21]To create a ray, we start by calculating a vector perpendicular to the main beam direction. To create a thick beam, we need to know which way is sideways relative to the beam's forward direction. Source stir. In 2D, we get a perpendicular vector by swapping X and Y and negating the new X. This mathematically rotates the vector 90°.

[00:00:38]Then, we shift the ray origin sideways along the perpendicular vector to give the beam some thickness. And finally, we slightly angle the ray direction using the perpendicular vector to simulate light spreading. Here's how it looks.

[00:00:51]>> [music] >> Physically, this beam represents a path of photons. And now that we have a ray, we need something for it to hit. First, we need the three points that make up our triangle.

[00:01:03]For the current pixel we are rendering, how close is it to the three line segments that connect our corners? We calculate the thickness for each edge, then add them together to form a complete closed triangle. If the current pixel is on one of those edges, prism edges will be a high value like one. If it's far away, it will be zero. We multiply a light blue color by that value and add it to the canvas. [music] So, now we have this variable glass object that will act as our prism. But, now we have to consider how a spectral renderer works and how it's even different from a regular renderer. When light hits a denser medium like glass, it slows down. [music] Because it hits at an angle, one side of the light wave slows down before the other, causing the whole beam to pivot. This is called Snell's law. Let's look at the boundary between two transparent media. A light ray arrives from point P hitting the origin. We measure the angle theta one from the normal line. In medium two, the ray bends based on index ratios. We can use this to calculate transmission vectors.

[00:01:59]Most graphics engines like Unreal or Unity store light as a three-dimensional vector. This is designed for monitors, not for physics. A spectral renderer discards RGB entirely, storing light as a wavelength in nanometers.

[00:02:11]So, you can see that the index of refraction is no longer fixed. [music] It changes with the wavelength. So, the combination of Snell's law with a unique index of refraction for each ray means that every wavelength bends by a different angle.

[00:02:28]As you can see, Snell's law is working, but there's a problem. The light doesn't seem to be smoothly transitioning. It's broken up into four distinct colors. In order to have a smooth gradient or spectrum of light, we need to use something called Monte Carlo sampling.

[00:02:43]>> [music] >> To understand Monte Carlo sampling, we will try to understand calculating pi through a standpoint of probability.

[00:02:53]Imagine a square box with a circle drawn inside it. The circle has area pi r squared. The square has area 4 r squared.

[00:03:01]A randomly dropped coin lands inside with probability pi over 4. So, pi is equal to 4 * the fraction of coins that land inside [music] the circle.

[00:03:10]So, let's start dropping coins.

[00:03:19]As you can see, after 500 coins, >> [music] >> we get pretty close to the true value of pi.

[00:03:23]Now, here's the first look we're going to take at the rendering [music] equation. Now, I'm not going to expect you to understand this equation yet, but when we eventually do calculate it, you can see we can do a numerical approximation. But, can you see that this looks [music] familiar?

[00:03:35]When you sample uniformly between A and B, the probability density function [music] P is a flat constant. Since the total probability must equal 1 over that interval, the probability of hitting any specific [music] X is 1 over B minus A.

[00:03:47]Now, when we look at the general form of the Monte Carlo estimator, which divides by P, you can see that if we plug our uniform probability into that general equation, it reduces down to the average value of a function equation from calculus one. Or put simply, the foundation of Monte Carlo is just the mean value theorem. It's just the integral over the interval.

[00:04:05]So, in the shader, we loop through our [music] samples and generate a few random numbers for each one. We use the first number to pick a random wavelength and calculate [music] its exact index of refraction so the glass bends correctly.

[00:04:14]The other number slightly jitter the [music] light beam's starting position and direction before we fire that specific ray at the prism and add up the results.

[00:04:25]>> [music] >> I don't know, you stare at the math for hours, but seeing it actually disperse into a perfect rainbow is just very satisfying. Now, let's continue on to 3D.

[00:04:41]We start by shooting a ray from the camera and letting it bounce through the scene.

[00:04:44]>> [music] >> Two values travel with the ray, throughput, which is the surviving energy, and radiance, which is the collected energy.

[00:04:54]If the ray escapes the scene, it picks up skylight, added directly to the radiance.

[00:05:01]At each point, we cast a shadow ray. If it's unblocked, we add Lambert shading.

[00:05:07]>> [music] >> Indirect light randomly picks a new direction in the hemisphere and continues bouncing.

[00:05:15]>> [music] >> What we solved is the light transport equation that we saw earlier.

[00:05:20]Here's how it looks.

[00:05:23]>> [music] >> Already, it looks pretty cool, but the goal still remains to switch to spectral path tracing.

[00:05:33]So, for each sample, we pick one random wavelength.

[00:05:37]Then we convert wavelength to RGB weight acting as a spectral band pass filter.

[00:05:44]Each wavelength gets a unique index of refraction.

[00:05:50]>> [music] >> Reflectance also uses a per wavelength value.

[00:05:54]If you remember our light transport equation from before, basically all we're doing is making it depend on wavelength.

[00:06:00]>> [music] [bell] >> Now let's look at our trace function, which handles the core spectral path tracing loop for our glass and ground intersections. It tracks the individual wavelengths across each bounce, executing the ray mesh interactions, environment sampling, and reflectance calculations we discussed. It may look complicated, but it's just what we discussed earlier. At the end we return our radiance. And this radiance that we're returning is our L in our radiance equation.

[00:06:33]So I imported [music] it to cancer model so you could better see the radiance.

[00:06:37]If you look closely, you might notice that the noise is colorful now, and that the glass is almost producing a rainbow.

[00:06:42]This is pretty cool, but I noticed that it's missing a pretty important feature that real glass has, caustics.

[00:06:49]>> [music] >> I want to show you this.

[00:06:54]This is a lens I made in glass blowing.

[00:06:56]And this bright spot where it converges light is called a caustic. You can see that I'm trying to focus the sunlight to burn this plastic, and there's the melted spot. To replicate this effect graphically, we need to learn photon mapping. As you may know from my previous video, I've been trying to get into university, and I have. I got accepted to UCSD for this fall. And while learning caustics, I found out a lot about a professor who taught at UCSD until 2018. His name is Henrik Jensen, and he received a technical achievement Academy Award in 2004. [music] His foundational math and direct technical consulting were rapidly adopted across major visual effects studios, proving instrumental in rendering the lifelike skin of Gollum in Lord of the Rings, and in Avatar, as well as the hair in Tangled. He also co-founded the render engine KeyShot. Really quite the guy.

[00:07:42]And he also invented this process called photon mapping.

[00:07:45]So, let's try to understand how photon mapping works.

[00:07:49]So, path tracing shoots rays from the camera, >> [music] >> whereas photon mapping shoots from the light. First, we spawn a photon at a random position on the beam with a random wavelength.

[00:08:02]Then at each glass surface, we refract using our index of refraction.

[00:08:07]We only record the photon when it lands on the floor after passing through glass. [music] Then we write into an accumulation texture called the photon map.

[00:08:19]So, here's the function for that process we just [music] described. To sum up what we're doing, we're shooting rays from the light.

[00:08:26]The rays refract through our glass and then accumulate on the ground.

[00:08:38]>> [music] [music] >> So, as I move, you might be able to see the photons accumulating in the photon map.

[00:08:52]And we do have a pretty interesting-looking optical effect, even though it looks a little bit wrong. But there's a major issue. It's running at around 1 FPS.

[00:09:00]This footage is sped up by quite a bit.

[00:09:04]And this is a problem that I really just couldn't get my head around.

[00:09:07]I couldn't figure out any meaningful optimization that would allow us to have a spectral path tracer and be relatively quick.

[00:09:15]So, I had a choice.

[00:09:17]I could keep going and try to optimize the engine more or I could reconsider the engine altogether.

[00:09:25]So, I chose the second option and realized that I had to do something that I vowed never to do.

[00:09:31]Hardware ray tracing.

[00:09:34]You see, I have a MacBook.

[00:09:36]And already that is probably turning on alarm bells in your head for the mere thought of RTX. How is this Apple product supposed to hardware ray trace?

[00:09:44]Well, guess what? My MacBook is the very first MacBook to have secret RT cores.

[00:09:50]The only reason that we are not so familiar with these powers is that nobody develops for them until now.

[00:09:55]WebGPU has this very year added experimental hardware ray tracing support.

[00:10:00]That means my apprehension to purchase a thousand dollar GPU was founded in a lie.

[00:10:06]I already had what I needed. But there was a problem. My engine doesn't support the newest version of WebGPU, version 29. I programmed it with C++ headers and so to switch to the new version I would have to switch [music] the whole thing to C headers. Well, that was so jank to process that I considered something I had long ago decided I would never do.

[00:10:25]Rust. WebGPU is made in Rust, so developing it in C++ is already a bit of a translation.

[00:10:32]So I wondered, why not just do it in Rust? The problem?

[00:10:36]Here's some C++.

[00:10:38]>> [music] >> And this is the same thing in Rust.

[00:10:41]See how it makes you want to throw up?

[00:10:43]Me too.

[00:10:44]I just didn't know how I was going to not only learn Rust but make a whole engine in it.

[00:10:48]Let's start by writing a default simple quad. You might be able to tell that a lot of the boilerplate actually looks the same. And here's our quad.

[00:10:57]Then I had to figure out how to re-implement my 3D pipeline.

[00:11:00]I started by making a simple spinning cube. Since the graphics code is exactly the same, it was mainly a port. Here's what it looks like.

[00:11:08]Then I re-implemented our spectral path tracer and as expected the FPS made a massive jump from two up to 80 FPS, [music] which was the purpose of this port to begin with. And here's what that looks like.

[00:11:19]>> [music] [music] [music] >> Mhm.