Let's take a look at a technique for computing Global Illumination (GI) entirely on the graphics card. GI deals with the problem of letting light bounce around, just like it does in real life, so the light reaches nooks, crannies, and surfaces that are just generally not pointing straight at the light source.



For motivation, and also so people who don't like explanations can quickly stop reading, let's first check out some before and after shots :)

Here are a white and a blue sphere sitting on a red plane; the light is coming directly from above. As you can see, this is a typical computer image; every object seems to live in its own world. Although the spheres cast a shadow on the ground, the red from the ground does not bounce back up to the spheres. This bounce-back effect is usually faked with ambient light tricks, but since ambient light is what GI wants to solve, such hacks are no longer relevant. Hence, the bottom of the spheres is pitch black.



Now here's the same image, with GI enabled.



As you can see, the flashy red ground almost becomes a shining light in itself, illuminating the spheres from below. Further, the white sphere reflects some light onto the blue one, and vice versa (although blue to white is pretty faint). All of this happens entirely on the GPU, without any precomputation, and the scene can be entirely dynamic (lights, objects and camera may move freely).

Again, seen from a different angle; direct light only:



With indirect light:



So, how does this work?

A key observation that many people have already made, is that deferred lighting makes adding more lights to a scene so cheap, that you can easily add a few hundred or even thousand of them, and still have a good framerate. This allows us to do what artists have been doing for movies and stillshots since forever: instead of just having one big light above, they place many fill lights to bring back some life to the dark corners. See for example the images in this post by the Infinity team.

The trick, then, is to simply place these mini-lights at runtime, automatically, and presto, you have Instant Radiosity [Keller97].

So what we need is a way to figure out where the direct light goes into our scene, and put a fill-light there. This is actually pretty simple, if you don't worry yet about specular light and secondary bounces. Indeed, if we render the scene as seen from the light, then every pixel that ends up visible and lit from this viewpoint, is exactly a point in the world that just received light from this location.

Thus, when we render the depth values for regular shadow mapping, we add some extra rendertargets to store luminance, and also a normal. We augment the shadow buffer with some extra G-Buffer-like rendertargets to have a setup similar to deferred rendering, which in the literature for some reason is called Reflective Shadow Mapping [Dachsbacher05].

So, now the RSM tells us where the direct light goes, with what intensity, and what the surface normal is. The next step is creating a large number of point lights at the corresponding locations in the world. Using instanced rendering, we pass 1024 spheres through the vertex shader. The VS samples the RSM to figure out where to put this sphere, and how large to make it; points in the world that receive more direct light, will bounce around more indirect light, and so we place a larger sphere around them.

Here's an example to make this more clear. The black veins in the stone texture do not reflect much light, and so the point lights that lie on (or close to) them, are pretty small. Brighter patches have larger spheres. Note that all spheres are shrunk a lot so you can see what's going on.



From here on forward, it's just regular deferred shading, this time on the main scene. The spheres are treated as lights, world positions are reconstructed, albedo and normals are looked up, and you just add a 1024 extra lights to your light accumulation buffer, almost as a post-processing step.

Another example, this time from a more typical scene with our favourite bridge. Direct light and some ambient:



.. and adding indirect light; note how the sides now appear illuminated from below, as it should be if strong sunlight hits these bright stones:



Of course, since the spheres are now positioned entirely on the GPU, the messing around typical of deferred shading, such as taking care when the camera is inside a volume, or doing stencil buffer trickery to make performance not suck, now also needs to work without stopping by the CPU first.

Here is the sphere layout for the above scene; note again how brighter spots get a bigger and brighter light:



The texture seams, unfortunately, are in the art.

On a practical level, the best results seem to be when using a regular R^2 based falloff, and consequently making the spheres larger by the square root of the measure luminance. But since my engine is not based around physical values, all of this is guess work and so the above images can be made arbitrarily brighter and/or softer by tweaking things like this... which is probably a good thing for artists.

Now the astute observer will notice one big glaring problem with this setup: what about shadows? If we just throw a bunch of spheres at the screen, then we get nice light bounces, but all of them ignore visibility. In other words, if there was a wall between the white and blue sphere, the blue sphere would still receive light from the white sphere, straight through the wall. Of course, we could just treat every sphere as an omni light and generate 1024 cube depth maps... but you can guess it won't be very fast :) There is a better way, and that will be hopefully for Part 2. Stay tuned!

Additional notes;

1. When instancing spheres, every vertex in every sphere will pass through the VS, which will time and again sample the RSM buffers to figure out where to move the sphere to. Obviously a faster way would be to instance points, proceed as before, and then have a geometry shader turn them into full spheres. Since I'm still suspicious of the performance impact of using a GS I didn't try this.

2. How do you sample the RSM? The simplest way is in a 32x32 grid, which is what I use. If luminance can vary strongly, then importance sampling is better, but doing this on the GPU is slightly tricky. It does, however, also allow the optimization of skipping locations that do not have any other geometry nearby, by folding ambient occlusion into the importance criterium.