## Friday, July 18, 2008

### Dual-Paraboloid Variance Shadow Mapping

Edit: Added the video that I recently made

I have to say, I really like variance shadow mapping. It's such a simple(ingenious) technique to implement, but it provides such nice looking results. I haven't had the need to implement the technique before, but I'm glad I did. Last post we implemented dual-paraboloid shadow mapping. And those of you with a PS 3.0 graphics card were able to have semi-soft shadows with percentage closer filtering. But now when we get rid of the PCF filter, and replace it with variance shadow mapping, we can fit all the code inside the PS 2.0 standard. Anyway, on to the code.

Variance Shadow Mapping Paper + Demo

Building the shadow maps:

Variance shadow mapping is really simple to implement. First thing we need to change is to create either a RG32F or RG16F surface format for our front and rear shadow maps (instead of R32F/R16F). This allows us to store the depth of the pixel in the red channel and the squared depth of the pixel in the green channel. So our new pixel shader for building the depth/shadow maps is this:

return float4(z, z * z, 0, 0, 1);

Blurring the shadow maps:

Variance shadow mapping improves upon standard shadow mapping by storing a distribution of depths at each pixel (z * z, and z) instead of the single depth (as with standard shadow mapping). And because it stores a distribution of depth, we can blur the shadow maps. This would produce some funky/incorrect results if we were just doing standard shadow mapping with a PCF filter.

So, after we have created our depth maps, we will blur them with a separable Gaussian blur. This will perform two passes on each shadow map; the first will perform a horizontal blur and the second will perform a vertical blur. There is a wealth of information on the internet on how to do this so I won't explicitly cover this. Here's what our front shadow map looks like after being blurred:

We build our texture coordinates exactly the same as the previous method of shadow mapping. But the depth comparison is a little different. You can refer to the VSM paper for an in-depth discussion, but here is the gist of it. Since we filtered our shadow maps with a Gaussian blur, we need to recover the moments over that filter region. The moments are simple the depth and squared depth we stored in the texture. From these we can build the mean depth and the variance at the pixel. And as such the variance can be interpreted as a quantitative measure of the width of a distribution (Donelly/Lauritzen). This measure places a bound on the distribution and can be represented by Chebychev's inequality.

`float depth;float mydepth;float2 moments;if(alpha >= 0.5f){    moments = tex2D(ShadowFrontS, P0.xy).xy;    depth = moments.x;    mydepth = P0.z;}else{    moments = tex2D(ShadowBackS, P1.xy).xy;    depth = moments.x;    mydepth = P1.z;}float lit_factor = (mydepth <= moments[0]); float E_x2 = moments.y;float Ex_2 = moments.x * moments.x;float variance = min(max(E_x2 - Ex_2, 0.0) + SHADOW_EPSILON, 1.0);float m_d = (moments.x - mydepth);float p = variance / (variance + m_d * m_d); //Chebychev's inequalitytexColor.xyz *= max(lit_factor, p + .2f); //lighten the shadow just a bit (with the + .2f)return texColor;`

5x5 Guassian Blur

9x9 Guassian Blur

And there you go. Nice looking dual-paraboloid soft shadows thanks to variance shadow mapping.

As before, your card needs to support either RG16F or RG32F formats (sorry again Charles :) ). You can refer to the VSM paper and demo on how to map 2 floats to a single ARGB32 pixel if your card doesn't support the floating point surface formats.

1. Oh yeah, in todays branch-capable gpus, do you think this will run any faster?:

float lit_factor = (mydepth <= moments[0]);

if(lit_factor < 1.0)
{
float E_x2 = moments.y;
float Ex_2 = moments.x * moments.x;
float variance = min(max(E_x2 - Ex_2, 0.0) + SHADOW_EPSILON, 1.0);
float m_d = (moments.x - mydepth);
float p = variance / (variance + m_d * m_d); //Chebychev's inequality

lit_factor = max(lit_factor, p + .2f);
}

texColor.xyz *= lit_factor;

Do you think enough operations are being omitted for a benefit here? Off course it depends on the granularity of the shadows I guess, branches on gpus only really have effect when coherent across big patches. e.g. 32x32.

Cheers and thanks again for the wonderful post.

2. Hmmm I think the only way to find out is to do some testing. Like you said, gpus process pixels in batches, so even pixels that shouldn't execute the branch still would.

I think it would also depend on the scene also. If most of it was not in shadow, then maybe it would be of benefit to have conditional. Of course you could also switch to a different technique that does not shadow if beyond a certain distance on the cpu.

3. Very nice tutorial. It seems though that the light/camera generating the shadow can't move (at least the cast shadows stays in the center of the plane). Is this expected behaviour?

I would love an example with a moving point light:-)

4. You should be able to move the position of the light.

For mLightCamera, set the position to where you want it, and then for the target, make sure that it is aligned with the -z axis.

So something like this should work:

mLightCamera.LookAt(new Vector3(3.0f, 0.0f, 0.0f), new Vector3(3.0f, 0.0f, -10.0f));

5. If I move the light using entirely the original code, only changing the position:

mLightCamera.LookAt(new Vector3(8.0f, 0.0f, 8.0f), Vector3.UnitZ * -6.0f);

The light is correctly lighting that area, and it looks the like depth texture is created properly (the shadows are slightly stretched). However the shadows are projected wrong, as if the LightView or some other transformation is wrong. Im not sure what can cause this.

6. I mentioned in my previous post that the light needs to be aligned with the -z axis. So you would need to change the target so that the view direction is looking down the -z axis. Like so:

mLightCamera.LookAt(new Vector3(8.0f, 0.0f, 8.0f), new Vector3(8.0f, 0.0f, -10.0f));

7. Sorry, I found out my error after posting. Though it didnt fix the problem. After correcting the lookat position, its like the shadow "moves" with the light. So I tried removing the translation-part where -mLightCamera.View.Translation is multiplied with the mLightCamera.View, and that almost fixed it. Now one of the shadow maps is applied correctly (the one furthest away from the user camera starting position). But the second one is transformed wrong. Its almost as if the texture coordinates for that paraboloid just have to be inverted on the y-axis.

8. Hey no problem. I know there was a reason for the inverse translation but now I can't remember :).

Anyway, yes it seems that isn't needed. And the other problem that occurred from my translation from my development version to the sample, was that the front blurred depth buffer's pointer was getting reset to the backs on the second blur pass. This was the problem you were noticing. Just create a second blur component that blurs the back depth buffer.

I've uploaded a new version with the fixes. There was also a small shader change.

9. Very nice! It works now:-) Thanks a bunch. Im still trying to get my head around shadow mapping in general, and this example helped a lot:-)

10. Glad to hear. I think there are some good shadow mapping tutorials on ziggyware too.

Also, a book that I always recommend for beginners is Frank Luna's Introduction to 3D Game Programming with Direct X 9.0c: A Shader Approach.

It may not be for XNA but it is a really good book for learning the basics. I haven't read any XNA books so I can't recommend them.

11. Hey,

what I don't get is when you do:
float E_x2 = moments.y;
float Ex_2 = moments.x * moments.x;
And then: ...E_x2 - Ex_2...

Previously, you put in shadow map red channel the depth, in green channel the depth*depth, so E_x2=depth*depth and Ex_2 too, the only difference is that the multiplication occure in 2 different passes, how could the value change between Ex_2 and E_x2?

Thanks.

12. Hi Stef,

You're correct the shadow map contains [depth, depth*depth]. However this is then blurred and filtered. So when reading from the shadow map, you recalculate the squared depth to find the difference (or variance) between the filtered depth^2 and the re-calculated depth^2 squared.

You can have a look at the VSM paper that I linked to in the post.

13. Thanks for this fast reply.
It helped a lot :p
Btw, nice blog, I've discovered it searching about dual paraboloid mapping and I'll surely go back to see your work which is cool ;)

14. Thanks :) Glad I could help.

15. I've been trying to understand what your doing and I can to a degree but I'm in D3D, which is a left handed system and I think XNA is right. Anyway, I'm unable to get anything drawn on the depth maps (textures) and I think it's because of the different RH LH systems.