GSoC 2022: Many Lights Sampling in Cycles X (Weekly Report)

Hi everyone, this thread is going to be used for my GSoC project’s weekly progress report. I’m going to try to keep these posts as higher-level summaries, but I’m also planning to write longer, more technical posts on my personal website. Feel free to check it out - constructive criticism is appreciated!

If you have any feedback (questions, concerns, bugs, etc), please post it to this thread and I will try to respond as soon as I can!

24 Likes

Hi Jeffrey.
Awesome to see this feature implementation getting love again.
Will there be a branch for this project any soon? And I’d like to know if this implementation will speed up render times in scenes with multiple lights through volume objects( volume illumination)

2 Likes

Hi Roggii, thanks for reaching out! I’m pushing to a branch called soc-2022-many-lights-sampling which you can also see here. I’m also going to be periodically updating this diff (likely every week or major update or so).

As for your second question, support for volumes is definitely on the to-do list. Right now, this is the sequence that I’ll be trying to follow:

  1. CPU Implementation
    a. Point Lights
    b. Spot and Area Lights
    c. Emissive Triangles
    d. Distant and Background Lights
    e. Volumes
  2. GPU Implementation

This is relatively high-level and definitely subject to change, but this feels like a somewhat natural order for me.

7 Likes

Hi, I try to build your branch but it stop with:

Building CXX object intern/cycles/kernel/CMakeFiles/cycles_kernel.dir/device/cpu/kernel.cpp.o
In Datei, eingebunden von /home/user/blender-git/blender/intern/cycles/kernel/../kernel/integrator/shade_surface.h:17,
                 von /home/user/blender-git/blender/intern/cycles/kernel/../kernel/device/cpu/kernel_arch_impl.h:32,
                 von /home/user/blender-git/blender/intern/cycles/kernel/device/cpu/kernel.cpp:48:
/home/user/blender-git/blender/intern/cycles/kernel/../kernel/light/light_tree.h: In Funktion »bool ccl::light_tree_sample(KernelGlobals, const RNGState*, float, float, float, float3, float3, int, uint32_t, LightSample*, float*) [mit bool in_volume_segment = false]«:
/home/user/blender-git/blender/intern/cycles/kernel/../kernel/light/light_tree.h:192:1: Fehler: Kontrollfluss erreicht Ende von Nicht-void-Funktion [-Werror=return-type]
  192 | }
      | ^
/home/user/blender-git/blender/intern/cycles/kernel/../kernel/light/light_tree.h: In Funktion »float ccl::light_tree_emitter_importance(KernelGlobals, float3, float3, int)«:
/home/user/blender-git/blender/intern/cycles/kernel/../kernel/light/light_tree.h:93:1: Fehler: Kontrollfluss erreicht Ende von Nicht-void-Funktion [-Werror=return-type]
   93 | }
      | ^

Translation:
Error: control flow reached end of non-void function [-Werror=return-type]
192 | }

Is it to early to build for now?

Cheers, mib
EDIT: gcc (SUSE Linux) 12.1.0

1 Like

Hi, yes - right now, I’ve only been building on MSVC with just warnings on. Thank you for pointing that out though - I’ll try to build with warnings as errors, and let you know when it compiles for me.

1 Like

Closing this thread to limit this to the weekly reports. @Jebbly Feel free to open a second thread for feedback if you like.

3 Likes

Weekly Report #1 (6/13 - 6/19):

This week, I completed an initial CPU implementation of the light tree with support for point lights, spot lights, and area lights. I’ve also written a longer and more technical post if anyone is interested. The scenes I tested consisted of a single cube, no background light, and some light sources scattered around. Here’s an example with 1 point light, 1 spot light, and 1 area light - 8192 samples:

Overall, my implementation’s renders seem to converge towards the original implementation’s renders given enough samples. However, as you can see in the above image, there is definitely a lot more work that needs to be done in terms of adjusting heuristics to reduce noise. Optimizations are also going to take place once the algorithm is deemed stable enough.

Objective for Week #2:

I’d like to complete the following for next week:

  • Implement support for emissive triangles
  • Implement support for distant lights and background light
    • I’ll be joining the bi-weekly rendering meeting to discuss how to handle this (since they need to be handled outside of the light tree)
  • Read up on light sampling from volumes in preparation for implementing light tree traversal inside of participating media
17 Likes

Weekly Report #2 (6/20 - 6/26):

This week, I added initial support for emissive triangles, distant lights, and background lights. Unfortunately, due to the fact that we ended up changing the sampling strategy, I wasn’t able to test this as thoroughly as I would’ve liked. Given that there haven’t been many tests for either triangles or background lights, I may have to revert some of the changes I made to test each feature individually.

I also started reading up on light sampling from volumes, but I haven’t made too much progress on that end yet. This is also going to be continued, but the other features take priority.

Objective for Week #3:

I’d like to complete the following for next week:

  • Test and debug support for both emissive triangles and distant lights
    • Some changes may have to be reverted and then gradually pushed again (depending on how well the features work)
  • Continue reading on light sampling from participating media from PBRT
16 Likes

Weekly Report #3 (6/27 - 7/03):

This week, I cleaned up some of my work and performed a bit more testing with emissive triangles and distant lights. Here’s a more technical post about how all the different light types were implemented. Again, the tests I did were mostly simple scenes, like this one with 2 emissive triangles:
2triangles_test
As I did this testing, I realized that I still haven’t updated some of the multiple importance sampling logic (for indirect lighting). This will be the main focus of my work for next week. I also created a scene that I’d like to use to verify some of the importance heuristic calculations.

Objective for Week #4:

I’d like to complete the following for next week:

  • Fix the PDF calculations done for multiple importance sampling
  • Debug some of the issues mentioned in the feedback thread
    • In general, I’m going to try to start debugging bug reports sooner rather than later, so that I don’t introduce too many new features that will make it much more difficult to debug
      • It’s also good to know which things are actually bugs at the moment, compared to things that aren’t working just because they have not yet been implemented
14 Likes

Weekly Report #4 (7/04 - 7/10):

This week, I updated the multiple importance sampling calculations to account for the light tree sampling distribution. Here’s a slightly more technical post summarizing what I did. Other than that, I also made some improvements and corrections to the importance heuristic. I mainly tested with the following scene, which contains 1 sun light, 1 spot light, 2 emissive triangles, 1 point light, and 1 area light:
week2_test_34s
At 1024 samples with adaptive sampling enabled, my method took me around 34 seconds to render while the original method took around 39 seconds. The last thing to test for now is combining this with environment HDRIs.

Unfortunately, since I spent a lot of time debugging the MIS PDF calculations and the importance heuristics, I was unable to attend to the list of bugs mentioned in the feedback thread. This is going to be my first task for next week.

Objective for Week #5:

I’d like to complete the following for next week:

  • Resolve, or at least trace and understand, the bugs mentioned in the feedback thread
  • Test scenes with environment HDRIs
    • The importance heuristic for these lights will likely need to be adjusted
  • Continue reading about the many lights sampling algorithm in participating media
    • Sorry that this has been on the to-do list for so long, but hopefully I can get started with implementation after next week!

Since this report also marks some degree of support for most light sources (outside of participating media), please feel free to start sharing scenes for testing, benchmarking, etc. in the feedback thread! I’ll try to check them out when I get the chance.

12 Likes

Weekly Report #5 (7/11-7/17):

This week, I mainly worked on refining the importance heuristic calculations and light tree construction. As I mentioned in the feedback thread, some of these changes are not yet pushed because I would like to merge properly with the master branch first. This should be completed within a few days.

At this point, aside from visibility issues (which are inherent to the algorithm/paper so a separate solution may be incorporated), I’m relatively content with the implementation for now. Of course, there are a lot of optimizations that still need to be made, but I’d like to begin working on sampling in volumes.

Objective for Week #6:

I’d like to complete the following for next week:

  • Merge my current work with the master branch
    • There are some structural changes to Cycles that seem to be affecting the accuracy of the algorithm, so I’ll need to investigate further.
  • Begin implementation of the algorithm for volumes

I expect the work for volumes to take 2 or more weeks, but if anyone has any simple scenes with volumetrics, please share them in the feedback thread!

10 Likes

Weekly Report #6 (7/18-7/24):

Unfortunately this week, I wasn’t able to make much progress on the volume side of things. I realized that I still need a better understanding of how volume rendering works with MIS before I can confidently implement proper support. In the meantime, I mostly spent this week debugging the issues mentioned in the feedback thread, which included:

  • Issues with instanced light objects
  • Subdivided surfaces having too much importance
  • Not accounting for area light spread

There are also a few other issues mentioned, which I would like to further discuss with Brecht and Lukas. In this case, I will likely be dedicating next week to resolving these issues. I apologize for the delays in volume support, and I’ll try to make sure I have sufficient background knowledge to implement support properly by next week.

Objective for Week #7:

I’d like to complete the following for next week:

  • Talk to Brecht and Lukas about what to do for textured triangles and ray visibility. I’ll work on implementing a solution if we agree on one, otherwise, we may defer it for a later time.
  • Resolve the issue with the point cloud lights.
  • Resolve the issue with the area and spot lights with large radii. I’ll likely have to change the heuristics a little bit, so if anyone is familiar with the paper and has any suggestions, feel free to add them in the feedback thread.
  • Catch-up on my understanding of volume rendering to properly implement support for the algorithm.
9 Likes

Weekly Report #7 (7/25-7/31):

This week, I talked with Brecht about the issues that were mentioned in the feedback thread. The proposed solution was to implement our own version of adaptive splitting, which I’ve begun working on. My current implementation uses a stack to keep track of the splitting, and weighted reservoir sampling to handle the stream of candidate samples.

Right now, the “weighting” is just sampling the candidates uniformly, so I don’t expect it to work well on some of the scenes that have been previously tested. However, even this initial solution seems promising for some issues from the feedback thread. These two renders were rendered at 16 samples:

Large Spotlight Radius:

Area Light Spread:

There’s still noticeable noise but I suspect that it will be much better once we use proper weighting for the candidates. The idea, for now, is to potentially take an actual light sample to perform the weighting. I’d also like to see if visibility can be included.

Overall, Brecht and I also discussed that a stable and refined implementation of surface sampling should take priority. As such, I’ll likely dedicate the remaining weeks of GSoC towards stabilizing and optimizing this new technique. The volume support may have to come later, e.g. over the winter.

Objective for Week #8:

I’d like to complete the following for next week:

  • Implement a better weighting calculation for the reservoir sampling.
    • I’d like to first use a light sample, and then see if visibility can also be incorporated.
  • Investigate how to calculate the PDF for multiple importance sampling

Since this new algorithm did require some restructuring of the code, I suspect that a few things will be breaking in the near future. For those testing things out, please continue to let me know of any issues in the feedback thread! The reports are helping a lot in terms of debugging, etc.

13 Likes

Weekly Update #8 (8/01-8/07):

This week, I worked on implementing the light sample for the weighted reservoir sampling. I also implemented a separate computation to address issues with the spotlight sampling. These solutions seem to have helped certain test cases, but a few new bugs have been reported in the feedback thread.

Objective for Week #9:

I’m volunteering at SIGGRAPH so I probably won’t be able to work much this week, but I’ll try to check out the reported bugs when I have the time. If anyone else is attending, feel free to reach out!

9 Likes

Weekly Update #9 (8/08-8/14):

This week, I was volunteering at SIGGRAPH and didn’t get much done on this end. However, I did meet up with some NVIDIA folks to discuss some methods of computing better MIS weights. I also talked to some developers at the Blender exhibit!

Objective for Week #10:

I’d like to complete the following for next week:

  • Implement/experiment with some different MIS weights
    • First, I’ll need to change the way that the PDF is currently calculated. It’s currently going from the child to the root, but I suspect that it will be simpler and more efficient (now that we also account for adaptive splitting) to implement the bit-trail approach used in the paper and in PBRT.
  • Debug the issues mentioned in the feedback thread
  • Continue experimenting with the weighting heuristic. I’m mainly concerned about the fact that taking a LightSample itself may be introducing unnecessary complications.
13 Likes

Weekly Update #10 (8/15-8/21):

This week, I modified the PDF calculations used in the multiple importance sampling. As mentioned in the previous weekly post, this uses a bit-trail to keep track of the leaf node’s location and also accounts for the adaptive splitting. This change seems to have resolved some of the issues that were mentioned in the feedback thread.

I also addressed the other issues mentioned in the feedback thread. There are a few issues remaining, but I believe the solution will be to fix the splitting threshold at some constant value.

Objective for Week #11:

Since I’m returning to my university this week, I’ll probably be working on less technical stuff and more documentation. I’d like to complete the following by next week:

  • Talk to Brecht about fixing the splitting threshold. This seems to be the easiest solution to many of the issues brought up in the feedback thread.
  • Along those lines, there also needs to be a good sampling method used for the weighted reservoir sampling. Right now, it returns a weight of 0 if it’s outside a light source, otherwise it returns 1. This isn’t much better than uniform sampling in some cases.
  • Add some limit to the amount of splitting performed. For large scenes, it may take a really long time to sample lights if there’s no splitting limit.

Other than this, I’ll probably be working on documenting things (e.g. one last technical write-up about the adaptive splitting, adding to-dos, etc).

15 Likes

Weekly Update #11 (8/22-8/28):

This week, I cleaned up some of the comments in the code and added future to-dos to this task. I’ve also begun collecting scenes that work and don’t work with the current implementation.

Objective for Week #12:

Since this is the final week, I’ll primarily be working on writing up the final report. I’ll also spend some more time documenting future tasks so that it’ll be easier for me / other contributors to pick up again (it’ll likely be in the winter for me). Furthermore, I’d also like to test a few scenes that are more complex, just to see how they compare as of right now. If anyone has any scenes they’d like to test, feel free to send them in the feedback thread!

11 Likes

Weekly Update #12 (8/29-9/04):

This week, I wrote up my final report and did some final debugging + documentation. I also tested out some scenes - one of the good results is NVIDIA’s attic scene.

At 2 minutes, the original implementation renders the following:

On the other hand, the many lights sampling algorithm renders the following:

There’s already noticeably less noise in this render (my final report also has a render after 30 minutes, where the MLS implementation also generally produces less noise). That said, it seems like the Bistro scene has some more work to do now that splitting has been introduced. However, keep in mind that there should be lots of room for optimizations!

Future Work

As GSoC comes to a close (and I resume classes), I don’t expect to be able to do much work for a while. I hope I’ve provided enough documentation for other contributors to pick it up, which I also plan to do over the winter.

In general, I’m very grateful to everyone who has shown interest and supported this project. The Blender developer community went above and beyond my expectations, and the continued support really helped me out. Thank you all - I hope I can return the favor someday!

30 Likes