Fixing/improving the rendering output and channels naming scheme

First of all…sorry if this is not the place for this. If that’s the case, please point me in the right direction.

I would like to suggest some changes to the file output naming and metadata options that I find inconsistent, inconvenient or even broken nowadays. (sorry if anything that I write here sounds harsh, it is certainly not intentional, english is a second language to me).

I tend to use the compositor file output node to render the multiple layers that I usually need. However, if I save a new version of the file I’m working with, I have to go and change the basepath for all the layers, otherwise I end up writing over the previous version.

To me, there is 2 main issues with how the output of a file currently works:

  1. The naming of the file/layers/filepath/metadata is too constrained to what the program assumes is a logical naming scheme, but it is not always the case, since every production has different needs. It needs to offer more flexibility to be really convenient.

  2. Lack of consistency between the two ways of approaching it (compositor nodes vs properties panel’s output options).

1. MORE FLEXIBLE NAMING SCHEMES FOR OUTPUT RENDERS

At its current state, the blender does not offer the flexibility to define valid paths and naming configurations to be really useful when you have to generate multiple iterations of a blender file. The paths are literal (even if they can be relative to the blender file path, the subfile structure is still literal), you have to type in the name of the files/folders, or navigate to it. That behaviour is prone to human error and leads to overwriting valid renders when versioning up.

I think a better way to approach this would be to make it more abstract, allowing the users to put in “tags” (I don’t know the proper name for it, sorry), that could be defined per project/blender file (I think the preferences would be a good place to do it), and that at writing time woud get replaced with the actual information

2. LACK OF CONSISTENCY AND METADATA ISSUE

One example: In the file output node, you can rename the channels to accomodate a third program input requirements, like Foundry’s Nuke. But in the output panel you cannot do that, tha output channel names are hardcoded. And when dealing with multilayer exr, it causes issues.

One example:

If you render a simple rgba multilayer EXR with a bunch of channels in it, the “Combined” image, as it is named by blender, will appear black in Nuke, because it expects a channel named “rgba” instead.

Which is particularly upsetting, because the obvious solution would be to use the compositor instead and name the channels at will, right?. BUT! The output file node produces cryptomattes that are not recognize by Nuke because all the metadata is “mushed” together in a single list instead of multiple metadata entries, so you cannot modify them because you cannot access them individually

So you either get:

  • A) a black image with semi-usable crytpo (because the metadata is not fully compatible namingwise, so you still need a modify metadata node in nuke)

or

  • B) an out of the box viewable image, but with no crypto available (or, if it is, I couldn’t find the way, and so it would be, in the least, a convoluted way)

PROPOSED SOLUTION:
I’m going to begin with an example of this solution:

Lets assume we have a .blend named myFile_v01.blend

  • If you are using the file output node, you would have to put //myFile/v01/ in the base path for each layer, and then remember to change the version number when upversioning your .blend

  • If you use the output panel instead, you would have to write the literal path and file name into the output path field //myFile/v01/myFile_v01_myLayer_0101.png, and once again remember to change it every time or risk accidental overwriting.

What I propose is using a tag system instead:

//{filename}/{version}/{filename}_{version}_{viewLayer}_####.{extension}

This would be used in the compositor’s file output base path slot, or in the output section of the properties panel. Once set, the output would remain the same, no matter the versioning, path, etc. The file naming would stay correct, there would be no accidental overwriting, and the flexibility gained would be huge. It would also be implemented for the channels and, ideally, for metadata characters that we know are conflictive, so it could potentially solve the cryptomatte issue too.

MAIN TAGS

{full_filename}: the whole name is taken as a string

{filename}: just the string part of the name of the file, not taking into account the versioning part of it, as defined by the version :point_down: item

{version}: could be defined by the user as a set of characters (similar to a regExp, or simpler). Something like v####, that would be searched and extracted from the name of the blender file.

{viewlayer}: the name of the view layer that is being written

####: the current way of determining frame numbering

{channel}: would affect the names in the output panel channels name, and the inputs in the compositor’s file output node. At the moment of writing the image, the channels /inputs would be set according to the names specified in the preferences window for those channesl

{date}/{time}/others: some generic tags that can be useful in the naming scheme.

I am sure I have left some other useful tags out, but you get the idea.

This is how I imagine the preferences panel could be:

So the output file would look like this.


And you could duplicate this two nodes, change the viewlayer selected, and all would work, nothing would need to be changed, the second viewlayer would go to a different folder, with the same configuration.

Most of the information in the fields proposed are already used by blender in some way (the filename, the version detection when saving with the + button), although I assume they would have to be implemented differently, not just exposing it. But I think it is doable, and would give a lot of freedom and make the UX better for the user.
Also, regarding the newest or unfamiliar users, those parameters wouldn’t prevent to use it as it is now, and the default behavior set in the preferences could be the current state of things. So if you don’t know that is an option, it is no hassle, and if you know what you’re doing, it would be very liberating.

I know it is probably incomplete, and at best is part solution, because the generation of the metadata and its requierements is a part I don’t know enough to take a solid stance.

What do you guys think?

13 Likes

Agree. Having some kind of global variables would be awesome. In the meantime you can try Loom – I’ve implemented a basic system for the output path a few weeks ago to basically get the marker and camera name as part of the file name: https://github.com/p2or/blender-loom/issues/9

I think we can make it work for the file output node as well, just takes some time to debug. Cheers!

2 Likes

I will definitely try it out as soon as I can. It looks very cool,thanks!
And yeah, being able to get the same filenames between output file node and regular properties panel would be nice.

Any chance that the channel data can also get custom names? I’ve tried looking it up, but am fairly new to scripting and can’t figure out the blender structure. I ask particularly because being able to use rgba instead of Combined would be super useful

Just added the channel support for the globals. If you have the time, test and report back @txo.

1 Like

Ok, sounds exciting!. I will try to give it a look tonight if I can (I am currently in crunch mode at work)

Very thorough proposition. I’d love to see it implemented!

that would be super awesome! I’m working in a vfx company and exactly this is pretty slow in blender to set it up to work inside our pipline - it would fix all my problems for the clean output…

1 Like

Yeah. I think the render layers/output setting is quite unintuitive, convoluted and rigid to be useful in a production. I’d love to know how people at tangent and other studios have dealt with it (I imagine that they would have developed in-house tools).

For me, at the moment, is quite frustrating to render passes that have be used in a Maya-Arnold-Nuke pipeline but cannot output the same naming scheme or metadata that the rest of the lighting passes have. So much so that more often than not I end up also preparing the nuke file so the comp department doesn’t have to deal with them.

Hi, all. Below issue is about saving render outputs with compositor and therefore posting here as related to the naming problem.

I usually render EXR outputs as 16bit to save disk space on animations. That bit depth is enough for most layers. Blender’s Output Properties -> Output saves Depth pass correctly in same file as a 32bit layer, which is required for compositing. But, as Blender’s layer names are not following commonly used standard I would use File Output node in compositor to save layers as correct names. Just like in original post of this thread, but manually typed layer names at this point.

Problem is that File Output node seems to force all layers as 16bit when I use Half Float setting in EXR (including Depth and other 32bit layers). I am using same settings as I use in Output Properties. Does anyone know how renders saved with Compositor’s File Output could be saved correctly and have most layers as 16bit and necessary layers as 32bit? I don’t want to save separate image for necessary 32bit layers, as that defeats the purpose having all layers in a single EXR file.

Has anyone tackled this issue or know if this is considered to be improved?

I personally had never tried to save different bpc per layer into a single multilayer .exr on purpose, but I made a quick test and you are absolutely right. These images show the issue you mention

bpc_inconsistency

By default (saving through the main output panel) channels such as Depth get stored as Full float, even if Half float is the selected configuration, and I think it is like that by design.
The same file exported through the file output node stores depth as half-float, so I would consider it a bug. I don’t think it is related to this thread though. I think it is more related to the other one I opened no so long ago, that one is about the inconsistency in the metadata stored depending on which method you use to save the render (main output vs compositor), so I think it would be a better fit. But I would report it as a bug either way.

I will make a bug report about this. Now I’m preparing for current project 2 comp outputs with different bit depths to get renders for compositing.

Can I use your example image, so I can include it with description and a sample .blend file?

sure, no problem! :slightly_smiling_face: