GSoC 2022: Many Lights Sampling in Cycles X (Feedback Thread)

Feature & Design Feedback
1

Hi everyone, since the other thread is going to be dedicated to weekly reports alone, please feel free to use this thread in order to provide any feedback on the Many Lights Sampling project.

15 Likes
2

Although not a major concern at the moment, I thought I would point out something. Compiling the Many Lights Sampling branch of Blender on Linux with GCC results in some build errors:

/blender-git/blender/intern/cycles/kernel/light/light_tree.h: In function ā€˜float ccl::light_tree_emitter_importance(KernelGlobals, ccl::float3, ccl::float3, int)ā€™:
/blender-git/blender/intern/cycles/kernel/light/light_tree.h:108:1: error: control reaches end of non-void function [-Werror=return-type]
  108 | }
      | ^
/blender-git/blender/intern/cycles/kernel/light/light_tree.h: In function ā€˜bool ccl::light_tree_sample(KernelGlobals, const RNGState*, float, float, float, ccl::float3, ccl::float3, int, uint32_t, ccl::LightSample*, float*) [with bool in_volume_segment = false]ā€™:
/blender-git/blender/intern/cycles/kernel/light/light_tree.h:217:1: error: control reaches end of non-void function [-Werror=return-type]
  217 | }
      | ^

Looking online I found this about the warning:

This warning is similar to the warning described in Return with no value. If control reaches the end of a function and no return is encountered, GCC assumes a return with no return value. However, for this, the function requires a return value. At the end of the function, add a return statement that returns a suitable return value, even if control never reaches there.

Errors like these are just to look out for as you do further work on implementing the Many Lights Sampling feature.

1 Like
3

Also, Iā€™m probably going to make a bunch of reports over the coming weeks as I test out the Many Lights Sampling branch. Sorry if it becomes annoying.

Upon initial testing, I have noticed a few issues. Iā€™m not sure if you know about them already so I am documenting them here just in case. Keep in mind some of these issues may be a issue localized to my builds. And some of these issues arenā€™t ā€œhigh priorityā€, theyā€™re just something to note down to fix later.

  1. If you open Blender, then entered a rendered Cycles viewport, then enable Many Lights Sampling, Blender will crash.
  2. If youā€™re in the rendered viewport, then you disable Many Lights Sampling, the viewport will appear darker than expected.
  3. When in a rendered viewport (with Many Lights Sampling enabled), hiding and then unhiding lights can result in the sampling of lights to change in a drastic manor. Also, moving the lights around in the viewport can result in the sampling of lights changing in a drastic manor. By ā€œchange in a drastic manorā€, I mean it can look like the scene was lit by an entirely different set of lights. For example, there might be a cube lit up with primarily red light and a little bit of blue light. Hiding then unhiding, or moving one of the lights may end up with the cube looking like itā€™s lit up primarily by a blue light with a little bit of red light.
  4. In some configurations, the Many Lights Sampling algorithm can be extremely different from the expected result.

Here is a video showing off issues 2 and 4. You can find the file I used for testing here: Unexpected result in Many Lights Sampling.blend - Google Drive

2 Likes
4

Hi,

Thank you for the posts! I suspect that some of these results are being produced because the importance heuristic is not yet optimized. In any case, thanks for sharing the blend file - Iā€™ll test out the issues you mentioned once I get the chance (it may be a bit).

1 Like
5

Iā€™m sorry to bother you, but Iā€™ve been reading up on various things and I found a note somewhere about Many Lights Sampling (I canā€™t remember where though) and itā€™s been kind of bothering me.

Is the Many Lights Sampling algorithm being implemented into Cycles going to be temporally stable?

For example, if a light is moving throughout the scene in an animation, will the heuristic for picking lights be temporally stable enough to not introduce major shifts in lighting in the animation at reasonable sample counts (256-4096 range)?

3 Likes
6

This is a great question! The Many Lights Sampling algorithm does indeed construct the light tree based on the location of the lights. If this is not updated, then there may be issues in certain cases.

Fortunately, from what I understand, there is a way to tag the light manager to update the distribution between frames. We would want to do this if any of the lights have changed position or intensity in order to reconstruct the light tree with the correct information. This does imply that there would be slight additional overhead during construction time, but it wonā€™t impact the actual render time.

So to answer your question fully, the algorithm should be temporally stable (although I havenā€™t implemented the support for it yet).

3 Likes
7

I remember where I found the comment about temporal stability with Many Lights Sampling.

Nvidia was working on the Quake II RTX game mod. Itā€™s a ray tracing mod for the game Quake II. One of the issues they faced was they were working with 1 sample per pixel and many lights in a scene, of which most had no impact on the final result. And as such a naive light sampling technique would waste a lot of samples.

To fix this, they looked at implementing a Many Lights Sampling algorithm, but decided against it because either the addition of a new light (when a character fires their gun), or a moving light (the bullets the character fires are lights), or both, resulted in significant enough difference in the sampling of lights between frames to cause a noticeable flicker, and thus a temporally unstable image.

This is what I was talking about when I was asking if Many Lights Sampling was temporally stable. Would a moving light, or the addition/removal of lights cause significant enough differences in the sampling of lights between frames to cause a noticeable flicker? Or is that an issue with the implementation that Nvidia used? Or is it a issue with Many Lights Sampling and low sample counts (1 sample per pixel)?

2 Likes
8

Iā€™m still not completely sure if I understand your question, but changing light sources will definitely change the sampling distribution. Itā€™s possible that at very low sample counts, the next frame with a different distribution will be relatively different. That said, Iā€™m not sure if anything specifically related to this sampling technique would be the cause of any flickering (I would imagine low sample counts would also be an issue with the default implementation).

Iā€™m wondering if the Quake mod encountered issues because they had to deal with certain limitations, because I canā€™t imagine that itā€™s possible to reconstruct the light tree so often in a real-time scenario like Quake. Do you happen to still have a link to that comment?

9

This is probably the answer I was looking for. But I will provide you with the notes from Nvidia below just in case.

Note 1: Q2VKPT is the name of the Quake II RTX mod before Nvidia took over the project and made further enhancements.

Note 2: When they referenced a ā€œlight hierarchy modelā€, Nvidia displayed an image of the Many Lights sampling paper that you seem to be basing Many Lights Sampling off of: CoolUtils File Converters
Iā€™m not sure if they actually based their Many Lights Sampling off of that technique, but they did shown an image of it so I have to assume that is the case.

  • Using a light hierarchy model was the first attempt at solving for the complex lighting challenges in Q2VKPT (a standard process in offline rendering).
  • This model consists of a tree of light sources with stochastic traversal performed using random numbers through the light hierarchy.
  • This ended up being a nightmare for the GPU because you are maximizing how divergent your axes to the memory are.
  • The quality looked inconsistent. For instance, if a rocket flew through the scene, the topology of the tree completely changed, so everything would flicker.
  • The team ultimately went for a simpler solution, using the potentially visible set from Q2VKPTā€¦

This comment is found on this page: https://developer.nvidia.com/blog/path-tracing-quake-ii/
It can be found in ā€œPart 3: Path Tracer (8 Minutes)ā€. There is both a video (only the first 2 minutes is really relevant) and a written section underneath covering the same content.

It should be noted, they didnā€™t actually talk about Many Light Sampling much, other than that they had issues with flickering when testing it.

Iā€™m really sorry if Iā€™m introducing confusion. I may of not understood your previous comment. And I may of used incorrect terminology in previous comments.

3 Likes
10

No worries, thanks for sharing! I somehow havenā€™t come across this before, but it definitely seems like they were aiming for something similar to the algorithm at first. Unfortunately, as you mentioned, they donā€™t really go into too much detail about it, so I can only really guess a few things.

Iā€™m not exactly sure what they mean by the axes being divergent to the memory, but I am curious how they were able to reconstruct the light tree while maintaining ~60 FPS. Itā€™s possible that they may have adapted the construction to make it more suitable for a real-time application. Even if they didnā€™t, Iā€™m also wondering if their temporal filtering may have made the flickering even worse when the light sampling adjusted to changing lights.

Sorry Iā€™m not able to provide much insight on this (take my thoughts with a large grain of salt). At the risk of trivializing this special case, I do think that 1 sample per pixel is a bit extreme for any decent render, and itā€™s something where even the default Cycles implementation will produce a lot of noise. In any case, the implemented algorithm should be unbiased and thus temporally stable at reasonable sample counts (the goal is for it to render the same things as the default implementation).

6 Likes
11

Thanks for taking a look at this and answering my questions.

3 Likes
12

Looks like Nvidia is working on it ā€¦

Spatiotemporal reservoir resampling for real-time ray tracing with dynamic direct lighting:

Fast Volume Rendering with Spatiotemporal Reservoir Resampling:

3 Likes
13

Sorry, I was mainly talking in the context of Cycles where a 1 sample render will generally be pretty noisy for a large scene. NVIDIAā€™s work has been very interesting and is definitely something Iā€™ll be looking at in the near future!

1 Like
14

No problem, I hope my answers were somewhat helpful. If not, feel free to contact me again (on here or in blender.chat)!

15

I agree, but in Cycles defense - it can be fast. I mean like fast-fastā€¦ real time fast:

But everything comes with a price :upside_down_face:

1 Like
16

real-time is needed :slight_smile:

1 Like
17

No worries, Iā€™m still in the process of adjusting the importance heuristic and such, but some of the results are looking promising :slight_smile:

2 Likes
18

Is this actually Cycles, and if soā€¦how are you getting it so noise free and realtime?

19

You mean like with no-noise ? if is it, is a real question i want ask you too @Jebbly because with optix too i have a little noise xā€™) (i THINKS IS UE4 CYCLES)

20

Sorry, that isnā€™t my demo so I canā€™t provide much insight into it. As you mentioned, there does seem to be some integration with UE4. The no-noise aspect could also be the result of some denoiser being used.

On my side of things, I, unfortunately, havenā€™t gotten to testing any GPU (i.e. OptiX, CUDA, etc) stuff yet, but itā€™s definitely on the to-do list!

1 Like