Taking a look at OptiX 7.3 temporal denoising for Cycles

Can I be a bit cheeky and post my tutorial on doing temporal de-noising using motion vectors here ?

It might be relevant to the discussion as its a technique Ive used on several feature films with great success.

First doing spatial denoising then temporal by average or median 3 or 5 frames works very well for a lot of shots. But fails at specific areas.

Thought it might be useful for the discussion.

4 Likes

Now that ⚙ D11442 Cycles X: Add OptiX temporal denoising support has landed in master, has anyone here tested it yet?
I’m not a coder (not even Python) and I could get it to denoise single frames “by hand” and so far they look good. Can some Python-savvy guy post a commandline to denoise a whole sequence? :wink:

And while we’re here: Temporal denoising requires a motion vector pass which can only be enabled and written to EXRs if rendered motion blur is off. How would one use temporal denoising on motion blurred sequences?

Thats a very valid question… this I imagine goes to the render dev team, this may be the only situation where we may need to have vector pass together with motion blur.

What do you think @brecht ?

I will run tests with the temporal denoiser and post results.

3 Likes

The steps to denoise the animation are described in D11442:

  1. Change Output File Format to OpenEXR MultiLayer
  2. Check the “Combined”, “Denoising Data”, and “Vector” pass in the render layer settings
  3. Render the animation
  4. Ensure the OptiX denoiser is selected in Blender Cycles settings and run bpy.ops.cycles.denoise_animation() in the Python console (And wait as Blender will freeze while it denoises the animation)
  5. Done (the EXR images now contain the denoised image sequence)

Note: The part you might be confused about is bpy.ops.cycles.denoise_animation(). You might think you need to put in some parameters, but you don’t. If you just use the command bpy.ops.cycles.denoise_animation() it will work.

I will post a video showing what to do if you want it.

1 Like

Great, thanks for this info! This was the thing that I was missing. All the OpenEXR and passes stuff is pretty selft-explanatory. But I didn’t know that I can just leave the parameters empty.

I have finally finished the renders. It can be watched at the link below, or can be downloaded from the link below so you don’t have to deal with Google’s compression (Note: You may need to wait a bit for the 4k version of this video to process if you want to stream it rather than download it):

Some notes:

  • The renders were all done at 1080p and upscaled to 4k in Davincii Resolve to get more bandwidth in Google Drive.
  • I tested three different denoising setups. OptiX Standard, OptiX Temporal, and OIDN. Let’s explain each.
  • OptiX Standard is me going into Blender and turning on OptiX denoising with Albedo and Normal maps for the final render.
  • OptiX Temporal is making use of the new temporal denoising feature with the Albedo, Normal, and Vector render passes.
  • OIDN is OIDN 1.4 with prefiltering set to Accurate and making use of Albedo and Normal passes.
  • You will find the name of each render at the bottom of the screen. It will be something like 0.01 - OptiX Standard. The 0.01 is the Noise Threshold setting for that specific render as the sample count was set to 4096 and adaptive sampling was enabled to achieve a specific noise level.
  • As a side effect of adaptive sampling being used, the PMJ sampling pattern is being used.
  • In these renders I had Animated Seed turned OFF. I personally found that having Animated Seed off helped OptiX temporal denoising, so that’s why I did it. However it should be noted that adaptive sampling does kind of act like an animated seed. Although my testing was rather limited and it may in fact be better with Animated Seed turned on (I will do more testing).
  • The frame rate has been increased to 30fps. How noticeable flicker is with each denoiser will change as you adjust the frame rate. So keep that in mind for your own projects.
  • The version of OptiX used is 7.3. I know 7.4 exists, and it may produce better results, but I personally used 7.3 in my testing. I also believe 7.3 is the version of OptiX used by the Blender build bot.
  • The scene used for testing is a slightly modified version of the “Classroom” scene from this site: Demo Files — blender.org

There are more denoising setups that could be tested. Different prefiltering settings of OIDN (None and Fast), a complex multi-pass OIDN denoising setup in the compositor, NLM, NLM temporal denoising, temporal re-projection like Statix talked about, etc. I’m sorry if I didn’t test the specific denoiser you wanted to see.

I will try to run some more tests on more scenes in coming days and post the results here.

15 Likes

Hi, you put a lot of effort in to Cycles over the last months and I really appreciate this.
I guess many other user feel the same. :slight_smile:

Cheers, mib

3 Likes

An update on this post. I have tested with the Animated Seed setting on and off to see how it impacts temporal denoising.

Having the animated seed turned off did help with temporal stability, but it introduces structured patterns across frames in the denoising, most noticeable when using a lower sample count.

3 Likes

I rendered two more scenes. The Junkshop and Monster scene from the Blender demo files: Demo Files — blender.org

Some notes about the renders:

  • The renders were done at 3840x2160.
  • The sample count was set to 4096 and the noise threshold set to roughly 0.02 (Junkshop was rendered in Blender 2.93 to avoid a bug, so the noise threshold was adjusted to roughly match that of 0.02 in Blender 3.0)
  • Animated seed was turned off. Results will be different with it on.
  • I tested three different denoising options. OptiX Temporal, Standard OIDN, and Multi-Pass OIDN. Let’s explain each.
  • OptiX Temporal is the new OptiX temporal denoiser, making use of OptiX 7.3. The denoiser was given the Render, Denoising Albedo, Denoising Normals, and Vector passes.
  • Standard OIDN is OIDN 1.4 with prefiltering set to Accurate. The denoiser was given the Render, Denoising Albedo, and Denoising Normals passes.
  • Multi-pass OIDN is OIDN 1.4 with prefiltering set to Accurate applied to each render pass (E.G. Diffuse direct, Diffuse indirect, Glossy direct, etc) prior to merging them to recreate the image. The Denoising Albedo and Denoising Normals are used as the inputs for each step of denoising. I ran this test as I’ve seen a few sources online suggest this produces better detail preservation that Standard OIDN and I can confirm it in some scenes.

Note: I would recommend downloading the file to watch it in the highest quality:


Now, let’s talk about some of the results.

  • In the Junkshop scene I was primarily testing how OptiX handles the hair on the character. You can see in the close up on the character’s face that some of the hair is blurred with OptiX temporal denoising (when compared to the others). However, it IS more temporally stable than the other options.
  • In the video for the Junkshop scene I did a close up on a glass jar. In this specific scene the “structured patterns across frames” in OptiX temporal caused by having the animated seed turned off is quite visible. In this same scene you can also see a loss of detail in the cloth below the jar with OptiX temporal when the camera moves away from it.
  • In the Monster scene when looking at the close up on the bed, you can notice a loss in detail when comparing OptiX temporal to Standard OIDN. However, if you look above the bed at the purple cloak, OptiX temporal is more temporally stable than the other options.
  • In the last close up comparison of the Monster scene, I have zoomed in on a section of wall where I noticed a loss of detail in the OptiX temporal render. Just something I noticed and wanted to point out.

Overall, it seems OptiX temporal denoiser loses detail compared to the other options, including the non-temporal form of OptiX denoising, in favor of temporal stability. Which you prefer is up to you.

4 Likes

I’m probably going to run one more test comparing denoisiers unless anyone has anything specific they’d like me to test.

The test I am conducting is I’m comparing NLM Temporal denoising to OptiX Temporal denoising.

One thing I would like to note for people that have been following the tests I’ve done: My results are only applicable to the scenes I tested. There are many more test scenes with different objects, material types, movements, etc, that will have different results.

I got an NLM Temporal test done. It can be downloaded from the link below.

Some details about what was used:

  • The render resolution is 3840x2160
  • The sample count is set to 4096 with adaptive sampling on and the noise threshold set to roughly match 0.02 in Blender 3.0
  • Animated noise is turned off
  • The OptiX temporal denoiser makes use of the OptiX 7.3 denoiser with the Denoising Albedo, Denoising Normal, and Vector auxiliary passes.
  • The NLM temporal denoisier is set to make use of Denoising Data with 5 frames of temporal history. The Strength and Feature Strength is set to 0.5 and the denoising radius is either 8px or 24px as noted in the name of the render.
  • Both OptiX and NLM use the right denoising data for the denoiser (NLM prefers pre-filtered denoising data while OptiX prefers raw denoising data)
5 Likes

Have you tried with animated seed/noise on? Or what was the motivation to turn it off?

I have run tests with the animated seed off and on.
Having it on lead to greater temporal instability after denoising. There are “artifacts” that occur due to the animated seed being turned off, but I was personally willing to accept those artifacts in my test for improved temporal stability.

Try using low Samples, it’s difficult to see the difference when the samples are already high. Use samples of between 4 to 8 let’s see the real difference in denoising

OptiX 7.5 has been released with some denoiser-related news, of which I understand very little about what they are :sweat_smile:

With regard to the denoising changes, it seems two main things have occurred:

  1. Nvidia has implemented an AI upscaler into OptiX that can be run at the same time as denoising.
  2. Some new features and adjustments have been made to reduce memory usage while denoising, made temporal denoising easier to implement, and some other adjustments.

At least that’s what I interpreted the release notes to mean.

3 Likes

I wonder if the UPSCALE feature would be available for RTX cards only…

NVIDIA® OptiX™ Version 7.5.0

Graphics Hardware:

● All NVIDIA GPUs of Compute Capability 5.0 (Maxwell) or higher are supported.

Graphics Driver:

● NVIDIA OptiX 7.5.0 requires that you install an R515+ driver.
● Windows 8.1/10 64-bit; Linux RHEL 4.8+ or Ubuntu 10.10+ 64-bit

1 Like

Hi, just trying this out. I need to do it from my own EXR files generated with Blender’s file output node.

I’ve set the file output node as below so that it’s layers and naming convention match what blender usually outputs:

This is the comparison between the blender output on the left, and my exr on the right:

but it’s giving me an error:

  • bpy.ops.cycles.denoise_animation()
    Error: Could not find a render layer containing denoising data and motion vector passes

I’ve set blender’s output to the location and file name of my own EXR sequence prior to running the script, so I’m not sure what the issue could be. Is my naming convention incorrect in the file output node?