for past half a year, ShadowCatcher in Blender has remained in a state which makes it unusable in any serious production. It’s lacking in many areas which are absolutely essential for any meaningful use in terms of integrating CG elements with real life footage. Since on the Blender home page itself, ShadowCatcher is described as a tool to enable users to CG elements with real life footage…
…I am going to present a detailed breakdown of why it is not capable of doing so. I will be using V-Ray as the example for comparison, but ShadowCatcher is handled similarly in other production renderers too (Corona, Arnold, etc…):
1, Incorrect shadow color:
Cycles’ ShadowCatcher does not capture actual shadows and illumination of the scene (HDRI in this case) with their correct color. It just renders completely grayscale occlusion from the environment, that’s it. This is something that will never hold up in a production scenario where correct representation of environment captured on the movie set is required.
2A, ShadowCatcher is not used for secondary rays:
Correct ShadowCatcher implementation is supposed to take backplate or environment map, and camera-map it on the surface marked as ShadowCatcher. This way, reflections, especially on contact points appear to be correct. Cycles’ ShadowCatcher solution completely ignores it and instead simply renders the original material of the ShadowCatcher object.
Some people tend to argue that it is possible to perform the camera mapping yourself, and assign such cameramapped material to the ShadowCatcher geometry. This is wrong. ShadowCatcher needs to reflect a ShadowCatcher material, which is not available in Cycles. It is a special type of material, which does not have shading, so it behaves kind of like emission, but at the same time, it can capture shadows as well as bounced light from the CG objects in the scene. Reflecting a simple, camera-mapped diffuse material would modify the surface with the scene lighting which is already once “baked” in the plate, resulting in a wrong ShadowCatcher surface shading in both specular and diffuse reflections. CG objects need to reflect exactly the kind of special ShadowCatcher shader that eye rays see.
Due to the same issue as 2A, not only specular reflection, but also diffuse reflection produces significantly incorrect results, which tend to harm resulting realism. You can see what a big difference incorrect diffuse reflection makes on this example.
Yet another problem with issue 2A is that it hugely increases room for error, as the material on meshes designated as ShadowCatchers has large impact on the final result. A simple misjudgement of correct diffuse color can completely ruin the result. Example above shows ShadowCatcher which has a red diffuse material in both V-Ray and Cycles. In V-Ray example, red diffuse Color has no effect on final result, as diffuse reflection is correctly taken from the projected background.
3, ShadowCatcher does not catch diffuse bounces:
Yet another defficiency which severely compromises output quality is the fact Cycles’ ShadowCatcher does not receive any indirect illumination from the CG objects in the scene. Therefore, final results lacks illumination from emissive objects, but more importantly, any degree of color bleeding. This is obvious already on the example #1.
4, ShadowCatcher does not capture reflections:
Last but not least, Cycle’s ShadowCatcher does not even appear to be capable of capturing reflections, which is another element that’s crucial for any degree of production use. This may appear to collide with point 2C, which states that diffuse material property should be ignored. That is indeed the case, but it applies to diffuse component only, which should be acquired from the projected backplate. Reflection component should be used by ShadowCatcher to enable users ability to specify reflective ShadowCatcher surfaces (for example wet road).
Now, you may be asking “How in the world would it be possible to composite all this mess together?”. Well, it’s actually not that hard. Only problem here is that there is currently no image format which allows Alpha channel to be stored as RGB instead of Mono. Storing RGB data in Alpha is crucial for this to work. Other renderers work around it in a very simple way where they allow you to save RGB Alpha channel as another AOV/Render Pass. Once that is out of the way, this is how it works:
1, You store any multiplicative data in the alpha channel (Shadows and Reflection occlusion)
2, You store any additive data in RGB channel (RGB and GI bounces)
3, You take separately saved Alpha channel with RGB data, invert it and multiply it over the backplate. This gives you proper colored shadows and introduces holdouts for RGB data and reflections)
4, You then Add RGB channel with additive information on top. And just like that, you have an output that’s 1:1 identical to the Beauty pass, with all the colored shadows, GI and reflections.
One more note:
In most of the renderers, there are two ShadowCatcher use cases:
- Integrating CG elements with real life footage
- Integrating parts of CG scenes with other parts of CG scenes, or “rendering in layers”. (For example, adding another character to already rendered CG environment).
For use case #2, it’s important to preserve some aspects of current ShadowCatcher functionality. Namely, it’s important to preserve an ability to use original scene materials for secondary rays (diffuse and reflection) instead of using ShadowCatcher material for secondary rays.
For this reason, other renderers give you a choice. In V-Ray, this choice is named “Matte for refl/refr”. Right now, Cycles has just one switch “ShadowCatcher”, which defines if the selected mesh is treated as ShadowCatcher. So what we need is one more switch, which could be called “ShadowCatcher for secondary rays”, which would define that the mesh will return ShadowCatcher shader for the secondary rays, not just for eye rays. If it’s enabled, then it will acoomodate for the use case #1, if it’s disabled, then it will serve the use case #2.
Issues #1, #3 and #4 don’t need to be preserved, as their fixes will improve both use cases.
Once these fixes are implemented, Cycles can become a very capable tool for any VFX work where integration with real life footage is required. Right now, however, it unfortunately fails to do what it’s supposed to do.