EDIT: Some the the structure for Many Lights Sampling has changed in a drastic manner since I wrote this comment. Some of this information is obsolete.
The Blender foundation has employeed a new developer to help out with the development of Many Lights Sampling.
So I wanted to collate a list of bugs/issues/limitations with Many Lights Sampling here so the information is more organised and readily avaliable. Both for the new developer, and for users who are using the Many Lights Sampling branch.
Note: Whenever I include image comparisons in this post, I am comparing renders of equal sample counts.
Here is the list of bugs/issues/limitations:
Incorrect Spot Light Importance - RESOLVED BY SPLITTING
The issue:
When a spot light has a large radius, Many Lights Sampling will over sample the spot light (and under sample other lights) in areas outside the cone of influence of the spot light. This results in an increasse in noise around the spot light.
This issue is resolved by using splitting.
What’s causing it?
My assumption is that the bounding box representation for the spot light is less than ideal for a spotlight with a large radius. This results in the spot light still being assigned “high importance” in areas outside the cone of influence of the spotlight.
Incorrect Area light Importance - RESOLVED BY SPLITTING
The issue:
Area lights with a large size, and low “spread” parameter have the same issue as spot lights. Many Lights Sampling will dedicate samples to it outside the region of influence the area light has. This results in an increasse in noise around the area light.
This issue is resolved by using splitting.
What’s causing it?
My assumption is that the bounding box representation for the area light is less than ideal for a these types of area lights. This results in the area lights still being assigned “high importance” in areas outside the region of influence of the area light.
Incorrect light importance when using textures - MOSTLY RESOLVED BY SPLITTING
The issue:
When a light uses textures to dictate it’s output, the light can end up being treated as if it has a greater or lesser importance than it actually does in certain areas. This can lead to over sampling or under sampling of the given light and increase noise.
This issue is mostly resolved by using splitting.
What’s causing it?
In Many Lights Sampling we use a number of approximations. It seems that one of these approximation is assuming that lights are uniform in brightness, which isn’t true for textured lights and that’s what’s causing this issue.
Incorrect light importance with variable material visibility
The issue:
In Cycles we can adjust the material visibility of objects and lights. For example, we can make a light visible to reflections, but not diffuse materials.
At the moment Many Lights Sampling doesn’t handle this properly and as such still dedicates samples to lights that have no benefit for the current material.
Splitting can help to aleviate the issue, but it doesn’t bring the quality up to the same level as rendering without Many Lights Sampling.
What’s causing it?
Many Lights Sampling currently doesn’t handle the case where an object is not visible to certain materials properly.
Poor balancing between distant lights and local lights
The issue:
In Many Lights Sampling, the balancing between distant lights and local lights can sometimes be quite poor, resulting in over sampling or under sampling of certain lights and increasing noise.
What’s causing it?
When picking a light from the light tree, we start by deciding whether or not we want to pick a local light or a distant light. This is done by comparing the importance of an approximation of all distant lights and the importance of the root node of the light tree and deciding between the two based on that.
The issue with this is that the root node of the light tree is a poor representation of the contribution of local lights to localized areas of the scene. And as such, it can negatively impact the accuracy of the decision on whether or not we should sample a distant light or a local light.
The proposed solution I showed in the image above is this patch here: https://developer.blender.org/D16179
We pick a local light and a distant light then compare the two and make a decision on which to sample based on that.
Increased noise in interior scenes when using bright exterior lights - CAN SOMETIME BE REDUCED BY SPLITTING
The issue:
When rendering a interior scene with some interior lights and some bright exterior lights there can be an increase in noise compared to rendering without Many Lights Sampling.
Splitting can help aleviate this somewhat when using local lights. However it does not help in the case of using a combination of local and distant lights.
What’s causing it?
Cycles doesn’t know if there is a wall between the current shading point and the light it picked until the light has been sampled. As a result of this lack of knowledge about the occlusion of lights, Many Lights Sampling may dedicate higher numbers of samples to lights behind walls without know it’s a bad idea, and as a result it can increase noise.
Incorrect brightnesses on glass in certain scenes - NOW FIXED
EDIT: This issue has been fixed.
The issue:
Reflections of lights on glass materials in certain scenes/situations are incorrect (usually too dark).
Splitting can hide this issue in some scenes. But ideally the issue should be fixed at the source rather than relying on splitting.
A .blend file:
https://developer.blender.org/F13675382
What’s causing it?
In Many Lights Sampling we disregard lights that are behind a surface if the surface is opaque. However, if the surface is transmissive (like glass), then we don’t apply this optimization. This is not what’s causing the issue.
The issue is that there is a mis-match between the definition of what a “transmissive surface” is in the functions light_tree_sample() and light_tree_pdf() in /intern/cycles/kernel/light/light_tree.h (the same applies for the respective distant light fucntions in the same file).
In light_tree_sample() a transmissive surface is a surface with a shader that contains a transmissive component. This is correct.
In light_tree_pdf() a transmissive surface is a surface that we previously transmitted through. This is incorrect. But why is it incorrect?
If a ray hits a glass material, it has two paths it can follow. A glossy path (bouncing off the surface). Or a transmissive path (passing through the surface). light_tree_pdf() should consider both of these paths as “we’ve just interacted with a transmissive surface” for light_tree_pdf() to be correct. But at the moment, only the transmissive path is considered for this and as a result the light_tree_pdf() calculations are wrong for reflections off of glass in certain scenes.
A comment about this can be found in the file /intern/cycles/kernel/light/light_tree.h
/* TODO: this is wrong, needs to also be true if the previous bounce had a transmission BSDF but
* sampled a reflection BSDF. */
const bool has_transmission = (path_flag & PATH_RAY_TRANSMIT);
Weird patterns when there is a combination of lights in front and behind a opaque surface - NOW FIXED
EDIT: This issue has been fixed.
The issue:
In some scenes with some lights in front and some lights behind a surface, you may notice weird patterns of increased and decreased noise.
Splitting resolves this issue in some scenes.
What’s causing it?
In Many Lights Sampling we disregard lights that are behind a surface when the surface is opaque.
The issue is the calculation to figure out if a light is behind or in front of a surface is inaccurate when there are wide bounding boxes around the lights behind the surface, and it can end up being wrong a lot of the time resulting in lights behind the surface being sampled and thus increasing noise.
https://developer.blender.org/D16192 atempts to fix this issue.
Lack of Many Lights Sampling improvements when rendering volumetric materials - NOW FIXED
Edit: Volumetric support has been added in a recent revision of the Many Lights Sampling branch. It’s not included in the build I Blender I have linked below.
The issue:
Volumetric materials do not benefit from Many Lights Sampling.
What’s causing it?
Support for volumetric materials to benefit from Many Lights Sampling has not been added yet.
Blender crashing when enabling Many Lights Samping in certain scenes - NOW REDUCED/FIXED
EDIT: The test scene I included in this description no longer crashes with the latest iteration of the branch. As such, this issue has been reduced. I say “reduced” because there may be more scenes that crash that I personally haven’t found yet.
The issue:
In certain scenes, Blender will crash when enabling Many Lights Sampling. Here is a file I personally experience crashes with.
https://developer.blender.org/F13675703
Note about this scene:
It is a collection of 15 spot lights with a radius of 0. If the radius is greater than 0, I can not reproduce the crash. If the number of spot lights is less than 8, I also could not reproduce the issue.
What’s causing it?
I tried debugging it, but I couldn’t really figure it out. Something during the light tree construction goes wrong and Blender crashes.
Noise discontinuities when using splitting and low sample counts - A LIMITATION OF THE TECHNIQUE
The issue:
When rendering with Many Lights Sampling, and using a combination of splitting and a low sample count, you may notice weird circles in different areas of the scene. And the noise inside and outside of the ciricle is different. This is known as a “noise discontinuity” and according to authors of the original paper Many Lights Sampling is based on (Importance Sampling of Many Lights with Adaptive Tree Splitting), noise discontinuities are an expected side effect of splitting and usually disappears at high sample counts or when using something like adaptive sampling to target a certain noise threshold. It’s just that the effect is more noticable in Cycles due to how we do splitting.
What’s causing it?
When using splitting, there are regions of the scene that traverse down the light tree in different ways and sample different sets of lights. This results in differences in noise in different regions and is most noticable at low sample counts. This effect is pronounced due to the fact there are hard boundraies between different regions.
[details=““Kernel assert” and “illegal address” issues - NOW REDUCED/FIXED”]
EDIT: Some values have been adjusted and the kernel asserts and illegal address issues have been reduced. I say “reduced” as most of these issues stem from precision issues with how computers, and it can be hard to figure out exactly how these precision issues may manifest in a wide range of different scenes, making it hard to write code to account for these issues without testing various scenes.
The issue:
When rendering with a GPU backend (like CUDA or OptiX), you may experience errors like “illegal address in…”. Or if you’re rendering with the CPU in a debug build of Blender, you may encounter “kernel_assert”.
I personally encounter some of these issues with a variant of the “Classroom” scene. Depending on whether or not I’m testing on Linux, or macOS, with CPU rendering in a debugging build, or GPU rendering in a release build, I can personally come across some of these kernel assert or illegal address issues in different places.
And you will likely encounter these issues too. If you are a community memeber and experience issues like these, sharing the scene you have issues with may be helpful in assisting development and fixing these issues.
https://developer.blender.org/F13676505
What’s causing it?
Throughout the Cycles and Many Lights Sampling code, there are lines of code like this. kernel_assert(variable < 0);
These are primarily used for debugging. If the program reachs that point in the code, and the condition inside kernel_assert() is false, then the program will stop there and you can look at what happened with a debugger. Usually these are added to the code to catch situations where something hasn’t worked out as expected so developers can look into it and fix them.
Typically kernel_asserts are deactivated in release builds of Blender (what users are using), but from my testing, they continue to show up with GPU render as “illegal address” error messages. I’m not sure if that’s an issue on my part or not.
[/details]
Rendering artifacts with PMJ
The issue:
When rendering with the PMJ sampler and complex scenes, artifacts can occur. See:
https://developer.blender.org/T101356
I personally recommend that you use Sobol-Burley for complex scenes.
What’s causing it?
This topic is still a little bit confusing for me. There is discussion on the mater in the bug report for the issue:
https://developer.blender.org/T101356
Brecht: With many light sampling, we’re currently using a single 2D pattern for sampling lights. First we use one dimension of the pattern for selecting the light, then a subset of the 2D pattern for sampling a position on a chosen light. With many lights, that’s really stretching the sample out a lot.
It’s probably better to use a 1D pattern for sampling the light tree, and 2D pattern for sampling the light here. Ideally this would be a 3D pattern to ensure each chosen light still gets a stratified pattern. We could always use a 3D Sobol-Burley pattern just for light sampling regardless of the setting, and leave PMJ for the others.
There’s some alternative solutions possible, increasing divisions along one dimension only, or splitting the pattern along both dimensions some now. But probably not worth spending time on.
