Thanks for the in depth reply! That does make sense as to why it would do it in that way. I was actually testing it in a different scene which I can’t show here unfortunately. In the shot I have water with whatever background reflected in it already. I guess this is a more specialised use case but surely not uncommon but I would want where my cg object is reflected in the water to have that reflection be shown instead of the plate reflection.
I was accomplishing this before with a second render layer where my cg object was white emissive and then using the reflection in the water there as an alpha to composite. So it would be useful then perhaps for an alpha channel in the shadow matte where reflections of the sky/other shadow cathers are 0 alpha but normal objects are 1.
Just to make it clear, you have a video or image or whatever of the “real world” already that looks kind of like this:
You have some water and some objects are being reflected in the water.
What you want to do now is use the shadow catcher to add an object in that reflects in the water? Like the scene below?
The new shadow catcher should be able to do this. However, you need to match the shadow catchers geometry and materials closely to the objects in your “real life” footage otherwise you may end up with results like these (Notice the issues in the “water” of this scene):
Yep, exactly that. Currently it’s just multiplying and brightening the existing reflections. I was hoping to be able to use the shadow catcher in a situation like that without having to render separate masks with render layers.
That compositing is wrong:
The reflections should be solid and not allow the background to slip over the reflections, right now that shadow catcher is not working correctly.
I would say that reflections catched by the shadow catcher should be in a different render pass TBH.
This is the correct version (which AFAIK is the actual full scene):
So this is the kind of result I’m hoping to achieve, how i was previously doing it with a second reflection mask render layer:
And this is what the shadow catcher is currently doing:
Not sure what’s up with that artefact where it’s contacting the water.
The compositing is correct, it’s just that the shadow catcher isn’t producing the correct values and as such the final result isn’t correct. I purposely changed the properties of the shadow catcher so that artifacts like these would occur.
I personally agree. At the moment reflections from the shadow catcher are saved in the “Shadow Catcher” render pass which has to be multiplied over the “real life footage”. The issue is that with sharp reflections (and probably many other cases), the shadow catcher needs to do a lot to change the pixels from the “real life footage” into the pixels you’d expect. And if your shadow catcher is even slightly different from the real world, you get large visible artifacts in your final composited result like the black lines in the image shown above or issues like the background appearing partially in front of the reflection of the CG object.
Having reflections and probably refraction separated out into an “alpha over” pass rather than being combined into the “multiplication” pass will help, if not basically resolve this issue.
Maybe it can be suggested over at Cycles-X feedback ?
Edit: I’ve made a suggestion: Cycles-X feedback - #227 by Alaska
The main issue is that the reflection in shadow catcher objects are stored in the shadow catcher pass which needs to be multiplied with your footage. Because the change in pixel values and colour from “real life” to “real life + CG” are so large, it means you need to match the shadow catchers materials and shape to the real life footage basically perfectly otherwise artifacts can occur.
I would recommend that in the mean time you stick to adding reflections to your scene using the old method, trying to use the shadow catcher for sharp reflections is probably only really going to cause headaches until something is changed about how it works.
Your other option is to replace the water with CG water.
I like the new implementation of the shadow-catcher. It’s definitely a huge improvement from where it was. Mad props! However, there are some things that still bug me. Maybe objects could be given a “Shadow Emitter” tag of some sort, so that the shadow catcher only catches shadows and reflections from objects that have that tag enabled? That would remove the issue of other objects in the scene (such as walls used to rebuild a scene for accurate lighting) reflecting on the shadowcatcher, because theoretically the reflections from these objects already exist in the real life footage, and I need them accounted for in the 3D CGI that I’m adding.
If you look at that example if suzanne has the “shadow emitter” tag enabled, only the blue-ray should render on the shadowcatcher pass, and the yellow-ray should NOT.
Note: This is a thing explaining how in the exact scene you’ve provided, the effect you want can be replicated in the Cycles-X branch of Blender already.
In the example you’ve provided, it seems the “orange” “ground” is your shadow catcher, the purple monkey head is your CG object you want to add and the blue “wall” is an extra object supposed to represent a “real life” object.
In theory, because the blue “wall” is a real life object, it should also be tagged in Blender as a shadow catcher. If you do that, then the yellow ray you’re talking about isn’t saved to the shadow catcher pass, instead only the blue ray is. The green ray is saved to the “combined” past in Blender.
Updated post - I’ve installed 3.0.0 on my old i3 laptop and it still runs! The implementation of shadow catchers is now everything I hoped for. The only gripe is the documentation is a bit sketchy, I had to go on Youtube for some clues Blender Daily.
It is necessary to match the “fake” material of the shadow catcher to the “real” background image. This influences indirect light from the scene scattered, reflected or emitted by the shadow catcher. The trick to do this is to use the “object” texture coordinate of the active camera object to project the “real” background image onto the diffuse input of a principled shader on the shadow catcher. It’s a bit tricky to set up some nodes to do the mapping but once you’ve cracked it the shadow catcher is an exact match to the background - EXCEPT you still have to guess some more or less correct specular and roughness values etc then play with brightness and saturation of the projected image (it will be slightly too bright and not saturated enough because it effectively gets a double dose of illumination). It does not have to be exact as the multiplication in the algorithm handles direct illumination and the human eye is fairly forgiving about indirect lighting.
You can make parts of the shadow catchers above a certain brightness value emitters so that lights in the “real” background illuminate the fake scene.
Like this:
Well done!
Hello everyone, I searched this thread but could not find the answer.
It seems that the environment strength should not be set to zero when using the shadow catcher, can someone please explain why?
thank you
The environment strength can be set to zero and things will still work.
hello Alaska I was trying to understand why the light contribution of the spotlight on the shadow catcher pass goes away when the environment strength goes from 0.1 to 0. Nothing else changes in the scene:
thanks
The issue you’re experiencing occurs when you’re using the new shadow catcher pass and there are no “Shadow catcher lights” in the scene. Since the environment in you scene is the ONLY “shadow catcher light” in your scene, changing the strength to 0 results in there being no “shadow catcher light” in your scene.
This issue occurs presumably due to the limitations of math. (The shadow catcher pass is calculated as a pass you multiply over your “real life” footage. When you have no “shadow catcher lights” in your scene, then your real life footage is believed to be a black frame with values of 0,0,0. And as such, the shadow catcher pass can’t be calculated properly (and thus produces a blank frame?). This is because there is no math that allows you to do real life footage x shadow catcher pass = desired result in this specific case as there is no valid equation for 0 x something = not 0)
Okay so I enabled the shadow catcher checkbox on the spotlight. It is now considered to already exist in the original footage. Thus we get rid of the light from the spotlight on the ground, since that light is already present in the original footage, and we keep the shadows and reflections of the cg object that need to be added. So far so good. But when I set the environment strength to 0, the bright reflection of the cg object is gone, even though there now is a shadow catcher light in the scene?
To investigate this issue, I tried to re-create your scene in Blender. But I was unable to reproduce the issue you’re experiencing.
Are you able to share the .blend file with me so I can investigate it further?
The file has been shared with me. And looking at it, it was setup different to how I had set things up, hence why I couldn’t reproduce the issue originally.
The issue with the reflection disappearing when the background is set to 0 is the exact same issue as before:
This is because there is no math that allows you to do
real life footage x shadow catcher pass = desired resultin this specific case as there is no valid equation for0 x something = not 0)
Except, the 0,0,0 (black) part of the image is everywhere outside the cone of the spotlight once the world light is set to a strength of 0.
I see thank you for the explanation. Then the environment strength should not be set to 0 when using the shadow catcher.
The environment strength can be set to 0 when using the shadow catcher. It’s just that in specific scenes, like the one you’ve shown, setting the environment strength to 0 can result in cases where the shadow catcher is unable to properly function.
In Nuke, you can use the blackpoint of the grade to change the intensity of the shadow. In blender, I tried using a maprange node but it was clipping really bad and any math conversions will erase the color values of the indirect bounce, I also tried using it as an alpha but the math values became dependent on the type of shadow the shadow catcher has since the value “1” is arbitrary. Let’s hope they ass the alpha on the next build.