Compositor assets

[izo]

Motivation

Many Blender users currently perform 3D modeling and rendering but then export their renders for post-processing in other software. While this use case will always be supported, we would like to improve the compositing workflow, such that it’s easier and more convenient for users to do post-processing and compositing in Blender.

We identified several ideas to solve this problem (see #134214). This post is about assets —specifically, we’re looking for community feedback to help us identify which assets should be shipped with Blender.

Proposal

The new workflow should include an asset shelf with several categories as shown in the following demo:

Target users

Mainly Blender users who are beginners to compositing.
Advanced users should benefit from an asset shelf and presets to use as templates for further tweaks, but they are not the main target users.

Assets

Without going into implementation details, we would like to include the following assets (each point may have multiple individual assets):

• Camera & Lens effects:

  • Sharpen
  • Unsharpen
  • Vignette
  • Grain
  • (Sensor) Noise
  • Defocus
  • Chromatic aberration
  • (advanced) Glows and bloom
  • Camera shake
  • Lens flare

• Filter Presets

  • Film black and white
  • LA / nostalgia filter
  • Venetian blinds
  • Half Tone & Comic book filters
  • VHS
  • Sepia Tone
  • Color Isolation
  • Infrared look

• Environment effects

  • Heat displacement
  • Atmospheric fog
  • Wave warping
  • Liquify filter

More advanced assets:

• Procedural noise textures
• Color Management
  • Custom tone mappers
  • Gamma compression

Feedback

Which effects or categories would you also like to have included?

[thorn-neverwake]

@izo

The new workflow should include an asset shelf with several categories as shown in the following demo

Speaking with year (decades?) of use with other compositing applications - an asset shelf such as this would be a potential negative feature to me.

I make constant use of pull-down menus with filter entries, and even with presets I would prefer a presentation similar to how blender’s “Groups” appear as an option in the ADD menu. So, even with a bank of filter presets - definitely needs to not only be available via the asset shelf.

I do think the asset shelf presentation will be easily understood and probably used by newbies, which may be part of the goal (encourage people new to compositing, to USE compositing).

But, for power users already compositing in other software - convenient asset presets are the last reason they’d switch over to compositing in Blender.

[Rob_Ang]

I agree with making it easier for beginners (maybe building a layer system similar to modifiers, because beginners won’t use nodes anyway but at the same time continue to add low level nodes for advanced users).

Some useful nodes for beginners (but also for advanced users) can be:

Color:

-Exposure/Temperature/Tint (for photography users)
-Grade(black and white point, lift,gain,multiply,offset,gamma… any junior compositor learns to use it for color matching)

In general you could think about converting many existing color nodes into assets.

Matte:
-roughness mask (takes alpha as input and “breaks” the edges, useful for stylizing but also to help blending in a comp)
-edge detect

Blend:

-Lightwrap
-Emboss
-Shadow

Filter:

-High pass filter (like Photoshop)
-Glitch
-Dirty

Transfomr/Distortion:

-3D card (“transform 3D” but in 2D, more precise than corner pin for perspective effects)
-Bend
-Pinch
-Swirl
-Polar Coordinates (from and to)

More advanced assets:

-Lens Distortion (a version of the node independent from the movie clip editor but with the possibility of using the same distortion models, perhaps less useful for beginners).

-Relight (with image, normal and position/depth input)

-Lin2Log\Log2Lin

Off topic: Some low level nodes to add, camera info (like geonodes) objects node (like geonodes but with the exception of the geometric output), solid/constant primitive (with the possibility of choosing color and resolution), color matrix, convolve 3x3, repeat zone, 2D lattice deform (this last one would be complicated and inefficient to recreate as an asset at the moment).

As for the temporal effects I have not mentioned them, will there ever be a support? (time offset, time echo etc etc or the possibility of accessing information calculated on a previous frame)

In any case it would be useful that in the nodes, even if designed for beginners, there is a panel (closed by default) with advanced settings for the effect.

[jonlampel]

@thorn-neverwake

even with presets I would prefer a presentation similar to how blender’s “Groups” appear as an option in the ADD menu.

This is already the case for compositing node groups! Works great.

I have an add-on with reasonably advanced versions of some of these effects already and would be happy to contribute some if desired. Many couldn’t be assets because of the limitations of not having inputs for all settings, but since that is changing in 4.5 I will be converting most of them to assets soon anyway. When enum sockets come to the compositor then they can all be assets and it will simplify things considerably.

[thorn-neverwake]

@jonlampel

This is already the case for compositing node groups! Works great.

Right, just saying I’d like to keep that as new things are also explored.

[izo]

Thanks!

@Rob_Ang

In general you could think about converting many existing color nodes into assets.

I agree, though technically they wouldn’t be called assets, but somehow exposed with previews and easier drag and drop concept for better discoverability than the current text based search.

@Rob_Ang

Blend:
-Lightwrap
-Emboss
-Shadow

Filter:
-High pass filter (like Photoshop)
-Glitch
-Dirty

These sound reasonable to me

@Rob_Ang

-3D card (“transform 3D” but in 2D, more precise than corner pin for perspective effects)
-Polar Coordinates (from and to)

Not sure if these should be assets. It might be better (or just more practical) to have them as core nodes.

@Rob_Ang

it would be useful that in the nodes, even if designed for beginners, there is a panel (closed by default) with advanced settings for the effect.

This is part of the plan indeed. Node groups currently support panels so high level assets should use them also.

[izo]

@jonlampel

would be happy to contribute some if desired

Thank you! It seems like many compositor addons implement some version of layer based compositing. We are also considering implementing that in Blender.

Do you mind sharing which color corrections / effects are the most popular among your addon users?

[brecht]

Personally I’d be inclined to have nodes assets with more inputs rather than small ones. For example noise and grain could be combined, sepia tone could be part of a bigger color manipulation node. VHS and Infrared look I’d probably leave out for being more demos than practically useful.

Gamma compression (correction?) seems like a legacy thing that should not be recommended to be done as part of compositing, but left to color management.

I wonder if closures could somehow be used to layer effects. We’re considering them for light filters in light shader nodes as well. But not sure yet if it’s a great fit.

[jonlampel]

Sure!

With a caveat. The biggest thing users get it for is being able to do corrections that happen before, between, or after their chosen color transform, which is a pain in default Blender. Working this way also makes some things appear broken (render previews, material previews, certain VSE workflows, etc.) since Blender wasn’t originally designed for users to manage the color transforms themselves.

That’s a different topic in a way, but it really changes what node assets are possible.

For example, many users like the simple ‘Blacks, Shadows, Highlights, Whites’ slider controls. That’s not exactly hard to build, but it would be fairly pointless to add it from the assets to a default scene where it would be applied before the color transform. The results would not be what users expect. So different effects need to happen at different places in the compositing chain and adding them in the wrong spot would make them appear broken even if everything is actually working as intended.

Whether or not “breaking” the colors in sRGB is good because it allows people to work with certain effects the way they are used to and have control over the exact output color of each pixel, or bad because it is lossy and makes trashing the image easy… is a bigger topic for another day.

But the caveat that the controls users like the most can’t always work pre-transform aside, the most popular non-default effects are:

• Sharpness
• Clarity
• Color Boost
• YUV Saturation
• Vignette
• Film Grain

Examples and explanation:

Compositing Assets Notes

[izo]

@brecht

Personally I’d be inclined to have nodes assets with more inputs rather than small ones

Agree. We discussed this as a general rule of thumb for higher level nodes.

@brecht

For example noise and grain could be combined, sepia tone could be part of a bigger color manipulation node

We have the limitation that node groups don’t support menus (enums) in the compositor, so not sure how to best implement this.

@brecht

VHS and Infrared look I’d probably leave out for being more demos than practically useful.

The use cases I know of include TVs or monitors as part of a scene. People tend to either use an external software for compositing or preprocess the video showing there in an external software

@brecht

Gamma compression (correction?) seems like a legacy thing

Right, we might need a better name but the intention is to use Gamma for artistic reasons only.

[Xorrito]

I think a great category would be animations. I made this Scroll Up node that translates in pixels per frame, (scrolling without jitter), specially useful for end credits and such.
Another useful node would be sharpening with the method called “Unsharp Mask” as a filter.

[izo]

@jonlampel

But the caveat that the controls users like the most can’t always work pre-transform aside, the most popular non-default effects are:

• Sharpness
• Clarity
• Color Boost
• YUV Saturation
• Vignette
• Film Grain

Thanks! Good to know this aligns well with the proposal for the most part. Color Boost would be an interesting addition indeed.

@Xorrito

specially useful for end credits and such.

We don’t have support for text generation in the compositor yet, so I wonder if this would fit best into the VSE in the meanwhile.

[OmarEmaraDev]

One of the things that is not clear to me is the apparent duplication between assets and built-in nodes. If the assert is essentially a built-in node with a slight improvement, then might as well improve the node itself directly. Having assets is not a goal, it is a means to an end.

For instance:

• Sharpen/Unsharpen → Filter node.
• Chromatic Aberration → Lens Distortion.
• Glows and bloom/Lens Flare → Glare node.
• …

So all those needs to be justified. If all they are is being more opinionated assets, then maybe they shouldn’t be in the official assets library.

[izo]

This sounds similar to the suggestion above of making regular nodes appear in the asset shelf (instead of adding a new asset). I think if we agree the effects should be in Blender, we can decide if a new asset is justified or not based on the implementation.

@OmarEmaraDev

If the assert is essentially a built-in node with a slight improvement, then might as well improve the node itself directly

Right, they shouldn’t contain a single node underneath with different default parameters. On the other hand, we want to move away from more complex nodes and build assets instead?

[nickberckley]

Not only I prefer the route of assets for the future, but I would split current Glare node into separate assets as well. I wish that path had been chosen instead.

1. Having effect as a node group you can enter and read the logic helps users understand how things work under the hood (to the certain extent, but to the extent that user is interested in) without dwelling into documentations. More built-in nodes get bloated with functionalities more difficult it gets to understand exactly what is happening, especially if the result of that node changes depending some circumstances that require beforehand knowledge.
2. Making slight modifications to the effects inside themselves is important. The only thing you can do now is try to chain nodes before or after the built-in node, try mixing stuff together to get something close.
3. If I want effect A, I’d much rather have one small node that just has inputs for effect A, rather than big one that has inputs for B and C effects that I do not touch.

Shader nodes is example of that already. Take Brick texture for example. It’s convenient, right? Yet you very rarely see it used in any of the actual complex setups. Reason is that it’s not customizable enough for users, and no matter what you mix after it you can’t get correct result you look for. It also lacks outputs that are necessary. If it was node group instead (and I wish it becomes one in the near future), I would just go in and make modifications, instead I have separate Brick textures node and saved as asset. That brings the forth point:

4. Contributing/updating assets is easy for artists, C++ code is not. If brick texture, nishita sky, etc. were assets I’m sure we would have many contributors improving them, modifying them, suggesting alternative variants. Instead they’re rarely touched.

I don’t think assets should justify being assets, I think it’s the opposite. Node should justify being complex enough that it can’t be an asset, and required C++ implementation, and developer who will maintain it and update it according to user inputs, which is lot longer and more difficult process.

[OmarEmaraDev]

I don’t think anyone disagrees with your arguments or the direction that needs to be taken. I just want us to be realistic, we already have a big Glare node, we have limitations on what can be an asset, and we need to improve the Glare capabilities in the compositor. In that case, my objection is to create yet another Glare asset instead of improving what we already have.

[Rob_Ang]

@OmarEmaraDev

Non credo che nessuno sia in disaccordo con le tue argomentazioni o con la direzione da prendere. Voglio solo essere realistici: abbiamo già un nodo Glare di grandi dimensioni, abbiamo delle limitazioni su cosa può essere considerato una risorsa e dobbiamo migliorare le funzionalità di Glare nel compositore. In tal caso, la mia obiezione è creare un’altra risorsa Glare invece di migliorare quella che già abbiamo.

Rather than duplicating, as already written elsewhere, in my opinion we should gradually replace the high-level C++ nodes with assets.

Indeed, in this sense, as a TD compositor I would like to contribute in some way to the growth of the blender compositor.

Last weekend I created a camera shake node with motion blur and autoscale functionality (to be improved in the next few days on high amplitude parameters). It works well using both CPU and GPU (I avoided vector blur and chose directional blur for motion blur) and is quite representative of what I mean by giving expert artists the necessary controls (under the advanced panel) and novice users the two main controls.

Here is the blend file

Blender Compositor

Feedback is welcome.