Cycles Manifold Next Event Estimation Sampling feedback

Hi

This topic can be used for discussions and feedback around ⚙ D13533 Adding Manifold Next Event Estimation Sampling Technique

There is a test build here.

Important: This feature is in early development, and not in master yet. The implementation may have bugs and limitations. This is a feature contributed by Reality Labs and developed specifically to render eye caustics. We are still evaluating how well it works as a feature for general rendering of refractive caustics in Cycles.

Just opening this topic, to keep the code review focused on development and give users a chance for feedback here.

13 Likes

Thx for creating the thread @ThomasDinges !
Thx a lot also to @olivier.fx (or Christophe Hery ?) for contributing the patch ! This is Fantastic !

It seem that MNEE can’t render caustic that lie outside of the shadow :

‘‘Note that the present formulation of MNEE’s implementation does not consider reflected caustics or refracted ones which lie outside the shadow of the transmissive object: this is because in these cases our method does not construct a seed path to start the process’’

An example render from @Caner-Aslan from the Patch comments :

Is there a plan to make the caustic goes outside of the shadow area ? Or is the MNEE a tad limited in that area ?

Thx a lot :slight_smile:


Edit 22 December :

This is know limitation. See all comments from Olivier Maury on patch differential.

‘’ 3. As reported, MNEE only tries to solve caustics inside the direct shadow areas of a light: This also comes from the way the solver is initiated (seed paths) and is also one of the issues SMS attempt to address.’’

1 Like

I’ll upload here the video test I did to compare it with LuxCore.

The reason to compare it with LuxCore is not to make a software comparison of “this one is better”, is more to show that the expected and correct refractive caustics result should deliver at least something similar to what Lux deliver with Light Tracing, but even if it’s not exactly the same, it shows clearly how there is a loos of energy in the caustics calculation.

It probably needs some improvement or modification to go into master, or it may not go to master as it is, who knows, but wanted to put that on the developer radar :slight_smile:

10 Likes

Indeed, Light Tracing in LuxCore AFAIK is like a lite version of Bidirectional Path Tracing, so I also believe that it is save to treat LuxCore result here as “closer to ground truth”

However MNEE can actually be better than LuxCore in some cases. This is actually incredible, in the scenes I tested, all of them are not as good as LuxCore results except for this one pool scene here:


I didn’t expect it to do it so well, MNEE in this scene amazed me.

7 Likes

That’s because Light Tracing does not work with SDS caustics, so only directly visible (reflective or refractive) caustics calculation.

This caustics don’t care about them being directly visible or SDS, they are just caustics (but refractive ones, and probably with other limitations :slight_smile: )

It’s not a matter of “better” but simply something not supported.

What I try to show is not if it’s better or not, but that I think there is a bug, some kind of loss of energy, in the caustic computation process :slight_smile:

1 Like

@olivier.fx thansk for the explanations!

Yes, I think your explanation says it all, and that must be the reason why we are not seeing more energy, so it’s just a limitation.

Out of curiosity, is it possible to increase limits for these limitations? Like giving the user an option to set more paths or bounces for caustics at the cost of performance? Maybe is a total limit of the technique or maybe is something that can be handled :slight_smile:

Also, now I understand the interface limit you mentioned :slight_smile:

Thanks! I’ll test the new version too.

I’m not super familiar with this at the moment, but considering the variables being added for using caustics, would this enable switching over to more advanced algorithms in the future / upgradeability, for example to SMS[RGL | Specular Manifold Sampling for Rendering High-Frequency Caustics and Glints], which appears to be highly efficient at caustics. This seems especially useful as the algorithm seems capable of support for both specular refraction and reflection, and potentially uses less memory.

4 Likes

This shouldn’t be hardware limited right? I’ve been trying to read up on the patch but I see nothing that says it shouldn’t work on Mac.

But while I see Caustic Caster and Receiver in the Object options, I don’t see Caustic Light under any Light on my Mac (latest 3.1 alpha and the D13533 tested).

Now that GPU/Metal acceleration is making it’s way, I really hope that general functionality like this isn’t tied to Windows, CUDA/Optix or similar.

Is it a bug that the Caustic Light checkbox isn’t available on my system?

no caustic light checkbox on windows 10 too… 3.1 patch 13533 from 20 december.
need reset factory defaults to show caustic checkbox.

It is something patch author mentioned himself:
https://developer.blender.org/D13533#360195

I would not expect MNEE to work that well as a general refractive caustic solution, i.e. for general high-frequency casting geometry: MNEE finds only 1 caustic path deterministically per sample, and for complex caustic casters, there are many solutions, as Tizian wonderfully shows in his paper introducing Specular Manifold Sampling (RGL | Specular Manifold Sampling for Rendering High-Frequency Caustics and Glints). We could certainly decide to extend this implementation to SMS but that’s currently out of scope.

4 Likes

awesome! It’s great to see such new papers being considered already for blender, even if out of scope for this particular implementation

1 Like

The devs. are using the same approach as all of the other big features from the last year. That is to get an initial implementation in first even if a little limited (ie. no feature creep), and then expand on it in following releases. Usually, a patch that becomes a monster stuffed with new enhancements can become a nightmare to review and test.

Cycles caustics are still buggy if you put water in the glass it disappears.

but separate work

3 Likes

Try scaling the water a tiny bit down.

I just thought I would comment here with what I believe is a bug.

When rendering with Shade Flat there appears to be a loss of energy. And when rendering with Shade Smooth there is some artifacts. Here’s some renders I did.
Note: Reflective Caustics are turned off and Filter Glossy is set to 0.

The file I used for testing can be found here:
https://developer.blender.org/F12888486

Shade Flat:

Shade Smooth:

I also tried rendering this scene by “brute forcing” it with Blender master rather than MNEE. And these are the results I got:

Shade Flat:

Shade Smooth:

2 Likes

Hi @olivier.fx thanks for the reply!

if you resize the water to not intersect the glass, you should be able to see caustics from both the glass and water.

No that didn’t work for me
image
Even if it does work, the thing is, I believe it is a standard practice to have water intersect with the cup.
Here is Blender Guru’s 2.80 version donut tutorial:

Look at the part starts at 12:40 , he talks about how the gap between water and glass is not correct, and how we should intersect the two to make it work.

So if MNEE does not work in this case, I believe it’s a bug.

I’m not seeing the issue of suzane over the pool killing the caustics

Are you sure the lighting is Nishita and sun height is 90 degrees? For me it’s 100% reproducible by by just opening the file and hit render view shading mode.

yes i got similar artifact, also water or glass in glass doesn’t work for me either.

One question after reading the thread on the developer site, is there anything that would prevent the algorithm from switching to a different seed path after X number of samples and adding the resultant caustic information to the accumulation buffer?

The algorithm itself would still be tracing a single seed path on every new sample put down, theoretically it shouldn’t even need to use the chosen path for the entire duration of the render because the caustics that are found converge so quick to begin with.


I’ve noticed that even if an object doesn’t use glass and has shadow disabled, it’ll still effect caustics


This is how the caustics look without the inner orb