Making sense of cryptomatte usage in third party programs

Ok, so here is one of my use cases:

Context

I work at a medium sized studio (currently around 70 people) that has a pipeline centered around maya(arnold)-nuke, and I am basically the only one of two people I know there that uses blender, at least on a regular basis.
We work in various tv series, so time and budget constraints are basically a law of nature, and in that scenario the studio cannot afford to take time from technicians and devs to deal with a software that is used marginally when there are lots of fires to put out already within the main pipeline environment. I imagine that the situation may not be some weird exception, given the gradual adoption Blender is experimenting throughout similar small-medium sized studios. What I meant is that I am basically working with what Blender offers out of the box, and some 3rd party addons, but no custom tools developed in-house.

Workflow example:

So, our renders coming out from Maya and into Nuke:
We usually separate the renders in 2 to 4 layers, depending the complexity of the shot. The composition usually follows this logic:

It is oversimplified a lot for the sake of clarity, but the green columns would be cryptomatte selection nodes (reading from the multilayer exr at the top) some of them with a selection already created in the preconfigured templates for the scene. Things like hair, shadows, AO, BG, characters, etc. all require different render settings regarding sampling, frame numbers, matte objects, shadow catching and casting, so they are renderered individually. Pretty universal basic stuff, really.

Besides that, often times there are more than one char layer when there are many of them in different terms of depth. One of the reasons is so the comp department can have better access to clean cryptomatte selection (to avoid edge artifacts that usually appear if some heavy compositing is done, due to AA differences in the borders between overlapping objects) , but also for scene optimisation. Whatever the reason, the cryptos have to be split too.

As you can see, such a workflow wouldn’t be feasible nowadays in blender, because you would need to have the renders separated via the compositor (which would mess with the metadata), and having it all inside a single .exr with A LOT of layers and passes would be a no go, for many reasons:if some layer fails all has to be rerendered, having to work with such all content of massive files every frame would kill the comp artists if they only wanted to preview the one layer, etc.

Another case:

Another problem is the incoherence between industry standard cryptomatte implementation and Blender’s. I have dealt with it several times when doing mattepainting work (for which I normally use Blender), because I have to provide a structure as close as possible to the one that Maya creates, so the comp department can work efficiently within a single comp file.

In the case of cryptos, it becomes an issue due to the compositor not recognising metadata. So I have to rerender manually (from the output panel) the minimum number of layers that I think we’ll need with cryptomatte, and create some custom masks if I need something else. In the image above, the matte input could be done in Blender, and maybe even the BG and FG if, for a particularly special shot, they are treated as a mattepainting. But chars would almost always rendered in Arnold, because they are integrated in the pipeline.

And, in terms of cryptomatte, it is kind of frustrating that being a industry standard, the implementation differs not only to that of maya in terms of . vs _, but within blender itself depending on from which panel the file gets outputted. Because all of that, I usually end up creating custom masks with emission shaders manually, but it can be a pain, honestly.

3 Likes

The cryptomatte manifesto not being written to disk through the compositor have been reported.
This is something we’re also struggling with, as it forces us to do a separate beauty render with lower samples just to spit out crypto from the panel instead of the compositor, landing us an additional 20% unnecessary rendering time pr shot.

That said, we have no issues with using 2.90 and 2.91 crypto output in Fusion or Nuke once its written correctly to disc.

Ps, deep support would be amazing, it would remove a lot of holdout / matting issues.

1 Like

Do you know of any way to make the crypto passes be readable in nuke without adding a modifyMetadata Node inside Nuke? I’d love to know about that, because even though the crypto passes themselves are fine and the data is correct, in my experience it does not recognize the name of the pass until you change the dots for underscores, and so the crypto layers don’t appear in the popup menu.Or do you modify the metadata externally beforehand?

1 Like

We updated to the latest version of Cryptomatte from PSYOPs github and we dont have any issues managing the layers or metadata manually.

Its just a readnode and cryptonode and its golden.

We’re on Nuke 11.2 i think.

1 Like

I hope it’s that easy. Thanks a lot!
Edit: just updated cryptomatte and you are absolutely right. It works like a charm. Thank you so much.
Although the other problems still apply, it really makes a difference.

1 Like

Hi all!

Yesterday during the render & cycles meeting we discussed about the main issue when using cryptomatte + file output node and want to include a patch in 2.92 that solves this issue without overhauling the complete system.

I just added a patch that solve the basic workflow issue. Any testing/feedback is welcome https://developer.blender.org/D10016

This change supports the basic use-case where the compositor is
used to output cryptomatte layers with a different naming scheme to
support external compositors. In this case the Multilayered OpenEXR
files are used and the meta data is read from the render result.

Meta data is found when render layer node is connected with the
file output node without any other nodes in between. Redirects and empty
node groups are allowed.

1 Like

Fantastic Jeroen!

Does this also work if there are multiple renderlayer nodes connected to the file out ? (but only one of them have crypto enabled)

We often have several passes that needs to be rendered (volumes, light categories etc) and we compile them into the same multi layer exr, where the beauty pas is carrying the cryptolayers.

sounds great, thanks! I will give it a try as soon as I can.

@JeroenBakker: Hello, great to see the improvment with cryptomattes and the file output node! I wrote the first bugreport about this, more than a year ago.
How can I test the patch? I´m not a developer and can´t compile blender.
Greetings Marcus

I updated the patch description with example files. In stead of testing it by building it you could also download the openexr file and see if it is can be use. https://developer.blender.org/D10016
Multiple viewlayer should also be supported. The system is setup generic so it could work with any render engine that support cryptomatte (RPR for example).

1 Like

@JeroenBakker: Thank you very much!
I testet your *.exr file with the compositing app fusion 16 and with Photoshop (exr-io plugin).
It works as it should! Great that the file output node now supports cryptomatte metadata!

Confirming that the EXR you posted works fine in Nuke. The file doesnt seem to contain the cryptoAsset layer but Material and Object works fine.

On a separate note, does this mean all of the metadata the panel offers will now run through the file out node ? Its missing lens, render time, node etc. All of the other metadata Blender writes when using the regular panel.

None the less, good job!

Yes, it works for me too. Just dragged it into Nuke, connected it to the crypto node and it works (asset is missing, but in the example above it can be seen that it is not connected to the output node, so all is good). Thanks a lot!

I can confirm that the *.exr also works with Adobe After Effects. I think so we tested the widely-used compositing applications. :slightly_smiling_face:

2 Likes

could not make it work in fusion
any fusion user here that made it work?
(blender 2.92 beta latest)
i exported object material and asset in a multilayer, then import in loader node then plug to crypto matte node(version 1.2.8)
it will only read object layer, if i try to touch the layer slider the entire cryptomatte node crash, not even the object layer will show again


@JeroenBakkerblender_OjHxG0p102

@Machieb said it worked for him, maybe he can be of assistance.
did you made the connection go through an empty node before feeding it to the file output node?
like in the example Jeroen provided.

Hello,
I tested the file Jeroen provided and it was working in Fusion, Photoshop and After Effects.

Today I made a cryptomatte test with blenders todays build with two of my current 3D projects.
In one project the cryptomatte works with the output node and compositing in fusion. BTW I don´t use the cryptomatte asset option. In the other project it works in one rendered view layer and in another view layer it´s not working. I don´t know why (objects, shader?), so my conclusion is, that it is not 100% working in fusion. I got a new workstation so I testet with Fusion 17 but cryptomatte plugin 1.26 installed via reactor.
What I see, in Photoshop and After Effects it works, mybe the cryptomatte implementation is different there?! I have no Nuke to test it there.
To answer the question about going through an empty group node. I don´t use these group nodes.


In that configuration the cryptomattes from the renderlayer on the right work in fusion that on the left don´t. :frowning_face:
But I wouldn´t blame the blender developers, I think the fusion cryptomatte plugin in fusion is not working correct?!

I just tested with latest buld (2.92) and it still works in nuke. The first image is from nuke and both cryptoAsset, object and material work fine for me.
cryptoAsset_result

In the end I didn’t use reroute or empty groups either and it works just as well.

As the icing on the cake, @p2or was kind enough to share a script he wrote to automatically connect the render layers to file output nodes (there is also a patch awaiting review I think). It is really useful to avoid typing in the names manually (it takes the node label as a prefix, wich is nice)

Hi all,
I need some feedback on some cryptomatte changes that have some consequences. See https://developer.blender.org/D3959

With the new design, the user selects the RenderLayer or Image from which the
Cryptomatte layers are directly loaded (the type of pass is determined by an
enum). This allows the node to automatically detect all relevant passes.

This reduces some flexibility when using cryptomatte layers that are not stored in the same way as blender does. We could still allow the old workflow to ensure this.

Then, when using the eyedropper tool, the operator looks up the selected
coordinates from the picked Image, Node backdrop or Clip and reads the picked
object directly from the Renderlayer/Image, therefore allowing to pick in any
context (e.g. by clicking on the Combined pass in the Image Viewer).

This reduces the need of the pick socket. When done right it could be removed at all.

A technical change is that the cryptomatte isn’t supported in node groups anymore. This is a legacy issue on how nodes store the references to scenes and images. Any feedback is welcome!

on builder.blender.org in the experimental tab you can find compositor-cryptomatte-workflow that includes this new workflow.

I keep checking the experimental branches page, but still seems it is not up yet (mac and linux are there, but I only use Windows). As soon as it is available, I will try it out