Game Engine Programming 088 - DirectX 12 Draw indirect aka indirect rendering

Added: 2026-05-15

[00:00:00]Hello everyone and welcome to the game engine programming series where we write a game engine from scratch. In the last episode, we improved material handling in the engine so that objects are rendered correctly when we assign materials to different parts of a mesh.

[00:00:17]Now we are ready to add the ability to actually assign textures and material properties to any 3D model in the editor. However, I noticed another issue that bothered me enough to postpone working on materials and fix it first.

[00:00:32]To demonstrate the problem, have a look at the frame rate in this scene. We can see that it's being rendered four times at a stable 60 frames pers, which is great. Now, let's replace the lab model by the Sponsa scene, which we all know and love.

[00:00:53]Now all of the sudden the frame rate has dropped to 15 fps. Granted, I'm running the editor in unoptimized debug build and this doesn't happen in release build, but it is still an indicator that something isn't quite right. So we have to fix this and therefore we need to guess what's causing it. At first one may think that this is because of the number of polygons in the spunza model.

[00:01:19]However, looking at the file size, we can see that the lab scene is almost 20 times larger. So, I don't think that the GPU is the bottleneck here.

[00:01:30]Next, we see that the Spunza model consists of a lot of sub meshes. Here, we can see that there are almost 400 sub meshes, and we know that each sub mesh results in a draw call in the engine. So in order to render four view ports, we are making about 1,200 draw calls, which could really choke the CPU, especially in debug build. To make it even worse, we are using two pass rendering, which doubles the number of draw calls to 2400.

[00:02:00]On top of that, we are also calling various API functions, which add even more cost.

[00:02:06]Fortunately, with modern APIs, there are several ways available for solving this problem.

[00:02:13]One way is to move draw calls to the GPU, which oddly enough is called indirect rendering. So when you tell the CPU to tell the GPU how to render, it's called direct rendering. But when you directly tell the GPU what to draw, it's called indirect rendering. Anyway, to do this, we have to create a command buffer. And instead of recording commands on the CPU, we let the GPU do it. Before filling in the buffer, we have to tell the GPU what's in the buffer. Therefore, we create a command signature. This is very similar to how we make a root signature. So, I'll put them close together. We'll implement this function in a bit.

[00:02:57]First, I'll add a helper class similar to what we have for root signatures.

[00:03:02]This class helps us fill in the indirect argument descriptions, which is similar to root parameter descriptions.

[00:03:10]In case of root signature they are called root parameters and for command signatures they are called indirect arguments.

[00:03:46]here. We can see all the types that can be used. Each command signature must end with a draw or dispatch argument in order to execute on the GPU.

[00:03:57]Next, we fill in how many arguments are in the buffer. This is done using a command signature description. Here I add a helper class for this as well.

[00:04:36]Now we are going to implement this function that creates the command signature from the description.

[00:04:43]We add the implementation right after the create root signature function in D3D12 helpers.cpp.

[00:04:55]As we can see, this is done simply by calling this API function.

[00:05:01]Okay. Now that we can define what commands can be in a buffer, we have to make a data structure for the buffer itself. We are completely free to decide what we put in the buffer as long as it agrees with the command signature that goes with it. Our command buffer contains the GPU addresses of the data that we sent to the shader. In addition, it contains the index buffer view of the sub mesh and the draw indexed argument which as I mentioned earlier must be used in order to execute the command buffer. We add two new functions that we can call for indirect rendering.

[00:05:39]Therefore, the old CPU path will remain available.

[00:05:44]Before filling the command buffer, we have to group our sub measures by PSO. I'll add a few data members to the G-PASS cache for this purpose. We'll see how they are used in a minute.

[00:06:10]We also need a pointer to the command signature for each sub mesh.

[00:06:21]We get this pointer as part of the material cache. So it needs to be included in this data type.

[00:06:29]Actually, let's make sure we create the signature before using it in the G-pass.

[00:06:34]This happens right after we create the root signature.

[00:06:41]So, let's add a new array that contains command signatures.

[00:06:46]We'll always have the same number of command signatures as root signatures.

[00:06:52]And like I said, we create the command signature right after creating the root signature.

[00:07:08]Again, this is very similar to how we fill in root parameters except we have two extra elements in the array. One for the index buffer view and one for the draw indexed argument.

[00:07:51]at the end. We used the helper class to create the command signature.

[00:08:05]And finally, we add it to the array of command signatures.

[00:08:31]We also release them at shutdown and clear the array.

[00:08:45]This should be an array of pointers, of course.

[00:08:51]Okay, now we get the command signature along with other material data in the cache. Going back to the G-pass cache, we resize the arrays we just added to match the number of render items and set the pointers within the cache buffer as well.

[00:09:14]In order to use a command buffer, we have to actually create a buffer resource on the GPU. This class will take care of that. Here we have a CPU accessible buffer resource and an array of actual commands that we fill in for each sub mesh to be rendered. Then we simply copy the content of this array to the GPU buffer.

[00:10:07]Note that this buffer is twice the size of the commands array because we use half of it for the depth prepass and the other half for the G-pass.

[00:10:35]As I said, the buffer is CPU accessible.

[00:10:38]So, we map it to system memory and remember its address.

[00:10:55]Then we add functions to upload commands to the first or second half of the buffer depending on render path. We also add a couple of accessor functions.

[00:11:10]Finally, we create an instance of this class for each frame buffer. Currently, we are using three frame buffers, so there will be as many buffers.

[00:11:19]Now, it's time to group render items by PSO. This way, we avoid changing the GPU state more often than necessary.

[00:11:28]In this function, we clear the flags and grouped indices.

[00:11:33]The purpose of the flags array is to mark items that already have been added to a group so that we can skip them.

[00:11:41]It's also used to mark the first item in a group that uses a different PSO.

[00:11:47]We use the lower four bits of the flags by for the G-pass and the higher four bits for depth prepass. So basically we use half of each bite also known as a nibble for each pass.

[00:12:00]The first bit in the nibble is used to flag grouped items whereas the second bit is used to mark the first item that's using a different PSO.

[00:12:10]The purpose of GPASS and depth grouped indices is to record the item indices within the cache that belong to the same group and therefore use the same PSO.

[00:12:20]Oh, So we run this code twice and for each pass we loop through all items until all of them have been assigned to a group.

[00:13:18]If we didn't exit this for loop that means that we came across another item that uses a different PSO. From this point, we mark every item that hasn't been marked before and is using the same pipeline state. We also add its index to grouped indices array.

[00:13:38]This algorithm is linear in case every object uses the same PSO and becomes quadratic if every object uses a different PSO. In most practical cases, I would expect the number of PSOs to be much smaller than the number of objects.

[00:13:55]So this should be reasonably fast. You could also use an unordered map to make this more linear in case you expect the number of PSOs to be large. At the end, we record the number of PSOs used for each pass.

[00:14:17]Next, we need functions that fill in the command buffer for each render pass.

[00:14:36]starting with depth command buffer, we get the index of each item within the cache.

[00:14:51]If the item is the first in the group to use a new PSO, we record the PSO's index as well.

[00:15:06]Then we simply fill in the command buffer for each item.

[00:15:33]At the end, we copy the buffer to the GPU by calling upload depth commands function.

[00:15:39]Recording commands for GPAS is the same except we also need to provide the lighting information.

[00:16:03]make sure that you change everything from depth to G-pass in this function.

[00:16:24]We release the command buffers when shutting down.

[00:16:28]Now we are ready to implement the functions for indirect rendering.

[00:16:35]In depth prepass function we call prepare render frame as before.

[00:16:41]Then we get the command buffer. Resize it to match the number of items and group the items by PSO. This will figure out the number of PSOs for both depth prepass and G-pass. We can use this to allocate a small array on the stack where we record PSO indices.

[00:17:00]Calling record depth command buffer will write to this array as well as record the commands. Obviously, at this point, we have enough information in order to render groups of items per PSO. For each PSO, we get the cache index of the first item that's using it. This way, we can get the root signature and PSO of the whole group.

[00:17:24]Also note that we use the same topology type for all items in the group.

[00:17:28]Remember that the primitive topology is part of the pipeline state subobject stream and therefore different topologies would result in different PSOs. So it's guaranteed that items within a group use the same topology type. We can calculate the number of items in a group from the indices at which each group starts. This index is in new PSO indices.

[00:18:00]Next, we set the root signature, the pipeline state, and the primitive topology. Then, all we have to do is call execute indirect with the command signature, number of commands, the command buffer, and the offset within the buffer where the command arguments start for this group.

[00:18:17]The offset is the index of the group's PSO times the size of command arguments block. Recall that we use the second half of the buffer for depth prepass commands. So in case of depth prepass the offset is with respect to the second half of the buffer.

[00:18:43]The function for GPASS is the same except we have to use G-pass data.

[00:18:58]Also, we use the first half of the command buffer.

[00:19:11]Okay, we can still build and run the editor using the old render path. All we have to do in order to use indirect rendering is calling the new functions in the 3D12 core.cp.

[00:19:23]I'd like to be able to toggle between the two rendering methods. So, I'll add a new compile time definition for turning indirect rendering on and off.

[00:19:32]So if this is set to one, we call the indirect functions.

[00:19:48]Need to move these lines before calling render functions.

[00:20:01]Okay, as we see here, nothing changed except now we are using indirect rendering. In order to prove that this is the case, I'll again replace the lab building with the sponsor model.

[00:20:14]And as we can see, we are still rendering at 60fps, which means that the CPU bottleneck is removed.

[00:20:27]And we also don't have any live objects after shutdown, which means that all resources have been freed correctly.

[00:20:35]It's always a good idea to enable GPU validation to see if there are any issues that we are not aware of. And sure enough, there is this error which actually doesn't have anything to do with indirect rendering. It's telling us that we are trying to index outside of the bounds of the resource heap where we keep our texture resource descriptors and we are apparently doing this in the post-process shader. So let's have a look there.

[00:21:05]Here we are trying to sample from the specular BRDF texture to render the skybox. But of course, we are not using this in the editor yet. So this index is invalid. I'll disable the skybox rendering for now till we have support for ambient lights in the editor.

[00:21:25]Now we get another error which does have to do with our command buffer and it's telling us that GPUbased validation will be less accurate when we change the index buffer view using indirect rendering. Of course, we have to do this in order to set the index buffer for each sub mesh. And I don't understand why it's this way. So, I need to research it a bit more.

[00:21:59]In the meanwhile, we can disable this particular error by setting a filter for it in the debug layer. This is done by defining an array of all error messages that we would like to disable and adding it to info's filters. There.

[00:22:40]These are not errors. It's just IntelliSense having special needs again.

[00:22:49]And now we don't get any errors anymore.

[00:22:51]And everything seems to be working as intended.

[00:23:07]Let's disable GPU validation again so it's not slowing down the renderer.

[00:23:19]Okay, that's it for today's video. Thank you so much for joining me and I'll see you next time.

[00:23:42]Hey, hey, hey.