As far as I can understand it, it’s a faster system that can be integrated into a classical unidirectional path tracer (like cycle) and work in team with the “default” system. It can help to generate caustic more quickly, help with Glints and seem well suited for an integration with the branched path tracer. Doing so, we could use a settings to specify how many bounces to use for SMS (0 = disable, 1 = only one refraction/reflection, etc…).
It seem that this solution is way better than the work on T38401 because it doesn’t require a long pre-computation step and can be integrated without being a separate integrator.
I know, it’s probably a huge task to do it right but I’m willing to help. I just got too excited and lost in the source code of Blender (and cycle). The source code is available for their algorithm and I already looked the “Multi-scattering caustic SMS”, “Glint SMS” and the combined path tracer. But my knowledge of the internal mechanism of cycle is the main turn down that keep me from trying to port a working system for now.
And, because this task is not as simple as adding just a magic button, I prefer to get cycle’s developers advises and comment before. It may be something that isn’t interesting (in that current form) for cycle. Do you thing it can be interesting ? Can be done ? What should I consider if you give me a positive feedback ?
Before adding something more specialized (and complicated) like this, I think it would be better to focus on path guiding. Path guiding is conceptually simple and while not as good at rendering caustics, it helps in more cases and is production proven.
There’s also an implementation of that available:
Note that in general Mistsuba code is GPL licensed, and for Cycles we need Apache licensed code. So we can’t copy implementations like path guiding or specular manifold sampling directly.
I’m not familiar enough with specular manifold sampling to know if it’s GPU friendly or leads to low frequency noise that flickers in animation (like MLT). In general these are things we want to take into account when adding rendering algorithms to Cycles.
More background on our plans in case you did not see it yet:
I am totally satisfied with cycles, and the caustics although useful, they are used few, or only for specific cases, before when I was looking for path guide in the new CyclesX, I found this, I don’t know but could be help. However, surely you are more informed about the state of research but in the case you have been lost this one, I share this link. but this seem more useful for real-time. Image-space Caustics and Curvatures.pdf
It doesn’t only help with caustic. It can help for transparent refraction/reflection to gain more quickly light informations with less samples. Like for defect on a metallic surface (see fig 12).
It may also help reduce noise in some case, helping to reduce the number of samples needed
@Tutul, reading the paper, it seems this has the potential be relatively GPU friendly and the (biased) variation is said to be temporally stable.
There are some challenges to integrate this seamlessly. In the current state of the paper it’s an effect you’d need to explicitly enable for some surfaces. Not something that can easily kick in automatically where needed.
Feel free to work on this and we can see if ends up being some practical. For me personally, it’s not something I can really dedicate time to now besides giving advice.
I guess refractive caustics with straight line initialization would be easiest to start with, since that could piggyback on the transparent shadow intersection code. Basically you’d record refractive intersections in addition to transparent BSDF ones, and if you encounter one of those you do a manifold walk.
Reflective caustics and glints would require more involved changes to the path tracing logic I think, so that’s something you might want to do when more acquainted with the code.
Advices would be really useful if I’m stuck in something. I’m glad you propose it.
I know about the current way it’s implemented and that’s not how cycle/blender work. I want to investigate either the possibility to detect caustic caster/receiver on the fly or just test a SMS path each time and find if it’s meaningful to keep it’s result.
On that note, based on the design goal for Cycle, I wonder if the biased or unbiased is a better solution to be included. Especially for that part : “Not all rendering algorithms must be physically correct, but we prefer to avoid approximations where we can.” What do you think ? Biased is clearly simpler to implement but come with additional settings that the user may want to tweaks. As I read about Cycle X it seem that you plan to reduce the amount of tweaking the user need to do ?
I remember proposing this idea to the developers and the main response I got is that integrating a bi directional path tracing into blender is not that hard. The issue is preparing the shader nodes to accept BiDir input to react with the rays properly. And the main issue they mentioned is cycles core code. Hopefully cycles X fixes this issue but as of now I am not sure.
The proposition here isn’t a bidirectional path tracer.
It’s an algorithm that can find a lot if not all valid path from a reflection/refraction to the current point. Starting from a valid path to find variance of it (with the help of a Newton iteration)
I know but in order to get MIS they have to get the shaders set up for at least BiDir before moving on to MIS since MIS and BiDir share more in common than path tracing and MIS. So if they support BiDir, moving to MIS wont be as hard as moving path tracing to full MIS
SMS is fully compatible with unidirectional path tracer. That was the main reason why I’m interested in that paper as it should be compatible with cycle without reworking how cycle handle the light path.
From what I saw, BiDir isn’t planned (yet) and Cycle X aim more for a guided path tracer before that.
@brecht (I’m reading the code for Cycle X) Blender has an option about reflective/refractive caustic and the code should know when the ray is a reflection or a refraction so I wonder if it’s doable to grab those information to detect “on the flight” and backtrack on the shape to retain it as a candidate for SMS calculation ?
I read the problem in the wrong way. The only way I see to find caustics caster would be to list every object with a reflective or transparent shader. I wonder if that could be done without too much work by doing SMS after the path tracer work but before denoising. During the path tracer “normal job” we gather the list of shape that caused a high reflection, or a refraction and use that list in a “SMS path”. But that would make each render longer as we need to sample each point twice (regular and SMS)
The paper seems to propose two methods for initialization: sampling points on all specular surfaces or taking BSDF samples to find hit points. That’s similiar to how emissive surfaces are already sampled, and imagine methods like MIS or light trees could apply.
In real world scenes most surfaces often have some specular component, so it’s not clear that sampling across all specular surfaces is going to be efficient. I imagine BSDF sampling or users manually tagging some objects could be more practical. Even then you have some type of quadratic complexity in the number of light sources x the number of specular surfaces.
Path guiding could be a good complement, guiding the sample directions for initialization and then using SMS to refine them.
Gathering a list of shapes that lead to caustics as you mention would also go in the direction of learning/guiding. To avoid double counting paths, you’d need to be able to identify and exclude those paths in the first iteration, or use a similar strategy like path guiding where you either throw away the first iteration or adaptively weight it with following iterations based on variance.
I personally think that the most efficient “first prototype” would be to let the user mark an object as a caster o the algorithm only need to get them from the scene. But, as part of Cycle design goal, it may be interesting to find a way for Cycle to automatically find objects that can be used.
Checking the source code I see that shaders have a bunch of boolean that help quickly detect which type of shader it is. Is it possible, without polluting the system, to just add one that can be determined based on the reflective/refractive element ?
To avoid huge calculation when the scene is complex, it may be interesting to simplify some part of SMS. After completing a path (single or multi-bounces) instead of linking it to 1 light we may instead like it to every light source before deleting those who aren’t visible. Then weighting each light based on their distance from the end of that path before doing the final SMS calculation.
Really look like we first need some guide tools for path calculation. It may be a requirement to avoid duplication or polluting cycle with “special case” code. It would be great to implement the guiding code as a tool that can be used either by the “normal” path tracer and be SMS (or maybe other type of algorithm that may come in the future).
EDIT: thanks for your imput and answer, it’s really encouraging
Could this work with some sort of “combined threshold” parameter that takes into account roughness of reflection/refraction, size of emitter and strenght of emission? While in a specified range activate the caustic sampling (photon emission?) below the threshold fade it to zero
I don’t know for now.
SMS will do additional sampling that will give you caustic (biased or unbiased method may be possible in the same implementation). A read your thread about photon emission and it look like it require a lot of work to get it working with the path tracer as it’s a completely other technique. And it’s probably not animation friendly.
Here the unbiased one isn’t too (well that may depend on some customizable settings) but the biased method (with or without additional constraints) is supposed to be stable and animation friendly.
Just to be sure it’s clear, my initial plan was to get help with that SMS implementation but to rework it from scratch as it’s not similar on how cycle handle a render path. It’s more for the estimator and the manifold walk that the code is really useful (I don’t have the required level to understand it 100% right and implement the big mathematics in the algorithm).
