Well, I will try to find why BPT is like cca 10x more efficient then current PT engine…but…until then, “future” is not efficient for me. Tried, Light tree…same bad result (end it adds more memory consumption, +time for generating it). It look like to me the problem is like for new W11 UI…more political, then practical…with results same…bad.
(I assume my problem is probably because BPT has extra tuning options “Sub samples” that probably globally overrides some “per material level rendering properties (like mat.->settings->surface/volume)”, and
that gives more drag to it in PT.)
When developing/changing some feature, there must be concrete reasons…mathematical and practical, not political like “will get extra time”. Mathematical reason is on “the paper one”. Practical reasons is…for example, when changing Cycles render engine, you should have 50+ scenes prepared, and re-render it, compare results…old render time, noise quality, memory consumption, etc…versus new results. This is what we programmers call “Unit Testing”…prepared scenarios where you execute functionality and compare results with expected values. I want to believe that was a case here…not as like someone in marketing was looking into optimizing schedules.
All, this gives me fear that one day someone will say…why Cycles, we have Eevee and ts enough…ditch Cycles…will make more coffee… lol … and currently, extrapolation is telling me that we are heading there.
For a Light trees…I suppose better rendering in indoor scenes, caustics, occlusions an etc, but with extra time, and memory penalty. It is good for one snapshot, but, I dont see how it helps when rendering animation with x1000 frames in it. Still it is good to have Light trees…and add new complexity to program, add extra time to service it, and in meanwhile, ditch BPT to optimize workflow.
Main question is who uses Blender, wide crowd that is trying to render coffee cup, to little play? Then, they ditch it and go surfing? For that crowd Blender is no use anymore because now you have AI for rendering such things… where you enter “I want cup of coffee…” and get quick results. That kind of users dont care about Light trees, PT, BPT … or any of these… and then why to focus on them…because they are many?
For me, my scene, AI is not clever enough, and, it wont be never clever+accessible enough. These are the users Blender team must fight for.
Most of the time the developers can’t test a feature until it’s programmed in. For example, with light trees, there were papers on it that showed promising results. So developers worked on implementing it, and then developers and users tested it in a wide range of scenes, and various optimizations and changes were made to achieve better results.
For example, Brecht wanted to avoid using ray splitting when using the light tree. But simply disabling ray splitting causes issues. So Jeffery, Weizhen, and myself all explored different ideas on how to avoid using ray splitting, and still get a good result. What we have now is a light tree without ray splitting, but without the issues disabling it introduces. However, as a side effect the light tree isn’t as efficient as other light trees typically are. But for many scenes, it still offers a benefit. See: Reference/Release Notes/3.5/Cycles - Blender Developer Wiki
I believe the Cycles developers prefer adding features like the light tree and ditching Dranched Path Tracing because overall, it’s a more efficient use of their time.
Let’s say we’re back in Blender 2.93. We have Path Tracing and Branched Path Tracing. Neither of these approaches are ideal for sampling many lights. Path Tracing samples 1 light at a time at random, that’s not efficient. Branched Path Tracing samples all lights at a time, and if the scene is expansive, or has a lot of localized lights, then Branched Path Tracing wastes a lot of samples on lights that have little to no impact in certain areas. So we need a better solution for the many lights sampling problem. Light Trees are an option.
So a developer starts work on implementing Light Tree support. And they have to effectively add the Light Tree twice. Because they can use ray splitting with Branched Path Tracing, but they can’t with Path Tracing. So they need to write two different techniques for how to use the Light Tree. As a result of having to write most of the Light Tree code twice, they’ve wasted more time programming, put less focus on refining both implementations, and has probably ended up creating bugs that are unique to both implementations.
If we look at it from the perspective of modern cycles without Branched Path Tracing, the Light Tree code only has to be written once, reducing development time, or allowing more time to be focused on refining the implementation. And there are fewer bugs, and it’s easier to test since there are fewer settings configurations.
So yeah, ditching Branched Path Tracing does reduce the complexity of Cycles, then adding features like Light Trees increases the complexity again. But without Branched Path Tracing, adding features like Light Trees is significantly simpler because the feature only has to be added once. And this isn’t just Light Trees, I’m sure some other techniques are like this too.
All these techniques like light trees and path guiding does increase render time, or scene initialization time, or memory usage. But work is being done from multiple people to reduce these impacts. For example, since the release of 3.5, Light Tree construction has seen a significant speed boost, and memory usage in some scenes have seen significant decreases.
An increase in render time and memory usage is bad, but the trade off from the developers perspective is worth it, and as time goes on, the concerns we have today about things like memory usage will become less of a concern due to advances in computer hardware.
Branched Path Tracing has been removed from Cycles starting with the Cycles-X rewrite.
Some people are bothered by this because Branched Path Tracing has various benefits over normal Path Tracing, so the removal of the feature made Cycles-X seem like a downgrade for people who regularly work with Branched Path Tracing.
But the removal of Branched Path Tracing has lead to other benefits. It reduces code complexity, it reduces the number of bugs, it reduces the time needed to implement certain new features.
This last one is important. By reducing the time required to implement some new features, it makes it easier for the Cycles developers or community members to implement new features that can improve the quality of path tracing. And in the process the addition of new features closes the gap in “noise vs render time” between Path Tracing and Branched Path Tracing, meaning the removal of Branched Path Tracing becomes less of a concern as Cycles development progresses.
Now, there will always be situations where Branched Path Tracing is better than using these other approaches, but the Cycles developers have made a decision. The benefit Branched Path Tracing offers in those situations isn’t worth the extra cost of maintaining Branched Path Tracing. Or they believe a feature can be implemented in the near future that closes the gap between Path Tracing and Branched Path Tracing in those areas. Or Branched Path Tracing doesn’t fit with the general design goals of Cycles now and hence why they didn’t want to add it to the Cycles-X re-write.
It’s possible that in the future Branched Path Tracing may return, or at least parts of it. But at the moment there doesn’t appear to be plans for that.
Note: Removing or Branched Path Tracing, or simplifying it, or merging some of the features into normal Path Tracing has been an idea that the Cycles developers have had for 6 or more years now. Hence why I believe that if Branched Path Tracing does return, only parts of it will return. See: #52725 - Branched path unification - blender - Blender Projects
use better hardware is nice…but while hardware goes up, scenes will go up…only optimized engine can get progress in quality & speed.
remove BPT to simplify, then do tons of developing to unify, where unification will never be the same as BPT, then you stuck in errors and bugs of new stuff…correct it and … in meanwhile I use old v293 BPT, and using new hardware finish the project. While I have done it, Blenderians will still trying to unify and fix tons of errors and maybe, get even worse result than before because didnt use unit testing & coz started with wrong assumptions after all. Sounds like a progress and ton of the advanced programming from a comics
Well, easier would be to branch & freeze developing BPT, just fix some critical error here and there, put notice and leave it on menu in Blender. Then you can experiment, play and loose time in a loop as much you want…
Not necessarily. Most people don’t needlessly increase the complexity of their scene because they have a better computer.
It seems what you’re suggesting here is the Cycles developers shouldn’t try replacing old features with new ones because the new feature they’re trying to implement may not be the best implementation, and they won’t know that until it’s finished.
If that was the case, then Cycles probably wouldn’t develop much and we wouldn’t get many new features. New features which do provide benefits over the old systems they are replacing. (E.G. Embree replacing BVH2, offering faster motion blur rendering.)
Yes, the Cycles/Blender developers can’t test features until they program them in. But they can figure out if it’s worth trying to implement based on the results published by other people, and how well the new systems are adopted by other render engines.
For example, the Cycles developers may think “We should switch over to using Embree. It offers many benefits over our own BVH system.” However since they haven’t implemented it, and thus can’t confirm it’s better in the case of Cycles, and they know for a fact that switching over to Embree will take time and is likely to introduce bugs, should they just avoid implementing Embree? No. Switching over to Embree will cause issues in the short term, but in the long term, switching to Embree is going to be an overall benefit for Cycles.
And we are already seeing this. Switching to Embree a few years ago has meant it was relatively easy for Intel to add Hardware Accelerated Ray Tracing support into Cycles recently.
I believe this is the same with Branched Path Tracing. Removing Branched Path Tracing and replacing it with other features will have negative side effects (bugs, inefficient code, increased memory usage, etc) in the short term. But it will have benefits in the long term.
The Cycles developers have unit tests, plus a collection of scenes they can use for testing new features and performance.
This technically already exists, it’s called Blender 2.93.
Yes it’s not the same as having Branched Path Tracing in modern versions of Blender/Cycles, but having Branched Path Tracing in modern Blender could come in various forms:
The feature gets programmed into Cycles-X. The developers do not think it is worth the time/effort, so they did not do it. It’s also possible that the design decisions made around Cycles-X that made it so much faster don’t lend their self to Branched Path Tracing. But I’m just speculating here.
Old Cycles gets left in Blender so people can use Branched Path Tracing with Old Cycles and use the new features with Cycles-X. This leads to a bunch of issues.
Old Cycles has a bunch of differences compared to Cycles-X. Many people will make bug reports about it, adding more load to the triaging team. This will happen even if there is a message saying “Please do not make bug reports about old Cycles”
Keeping Old Cycles in Blender will effectively double the size of Cycles. This adds bloat to the size of Blender that the developers do not want.
Keeping Old Cycles still adds a bunch of work on the developers part. Many of the libraries used by Cycles will be shared with Old Cycles. So when libraries for Cycles get updated, they will also need to be verified with Old Cycles, and in some situations changes will need to be made to old Cycles to accommodate the updated libraries, and these changes could lead to bugs they now need to fix. Another option would be to leave Old Cycles on old libraries and Cycles-X uses new libraries. But this is impractical for some libraries as some libraries will need to be updated to fix security issues.
Out of those two options, adding Branched Path Tracing to Cycles-X, then discontinuing it, seems like the better option. But it’s not an efficient use of time to do that. Why put in a bunch of time to add a feature, test it, bug fix it, etc, only to instantly discontinue it’s development as soon as it’s released?
If they’re putting in the effort of adding Branched Path Tracing, then they should put in the effort of maintaining it until they are 100% confident they can remove it and not negatively impact the workflow of their users. But they can never be 100% confident they can remove it. The Cycles developers could add Light Trees, Path Guiding, Manifold and Specular Next Event Estimation, Radiance Caching, spatial resampling, and a wide range of other features. And there will still be a collection of scenes that render better with Branched Path Tracing, and so removing Branched Path Tracing will upset the people that still use it.
So they’re stuck maintaining Branched Path Tracing for the rest of Cycle’s development, or they remove it and upset some people.
Obviously the Cycles developers have gone through thoughts similar to these. And have decided removing Branched Path Tracing with the Cycles-X re-write was the right choice for them. And yes it’s going to upset people and disrupt work flows. But that’ll encourage users to shift to the new work flow, a “standard” Path Tracing based work flow, that will only improve as time goes on and optimizations are made, or features are added.
And if users don’t want to shift to the new workflow immediately, they have Blender 2.93, which will get 2 years of LTS updates.
Also, I just wanted to show that the new techniques being implemented in Cycles can out perform Branched Path Tracing.
Expand to see full details
I have a simple city scene, it contains over 12,000 lights.
I rendered it with “Path Tracing” in both Blender 3.6 and 2.93 on the CPU with all the settings the same. 3.6 was about 5% faster. This is expected, Cycles-X didn’t see much of a performance boost for CPU rendering. Along with that, the noise level was practically the same to my eye. Note: Adaptive sampling was off.
So, I’ve established that the scene renders with roughly the same quality and performance on my CPU between both versions of Blender when settings are matched as closely as possible. I can now make a relatively fair comparison between Branched Path Tracing and Path Tracing with a Light Tree.
Branched Path Tracing had 1 sub sample for all material types, 8 samples per pixel. Sample all direct and indirect lights, 2 bounces. The render took 31 minutes and looked like this:
I then tested Branched Path Tracing with all the same settings as before, with the exception being that “Sample all indirect lights” was now off. The render took 18 minutes and looked like this:
Very similar to before in terms of quality and noise to with Indirect Lights, but rendered faster.
–
Path Tracing with the Light Tree rendered this image in 11 minutes, all other settings the same where possible (E.G. 2 bounces, no light clamping, etc):
A significant improvement in convergence. All while being significantly faster, and Path Tracing + Light Trees renders more samples per pixel resulting in higher quality motion blur and DOF when compared to Branched Path Tracing (This scene does not contain motion blur or DOF, but it is a benefit).
Note: There is a difference in brightness between the Branched Path Tracing and other images. Based on my observations this seems to be related to the amount of noise in the scene. Significantly increasing the sample count on both renders resulted in them both converging to similar results.
And for prosperity, this is normal Path Tracing, no light tree, rendered in 18 minutes:
Not all scenes are going to see the same benefit. But I just wanted to show that the new techniques being implemented can offer large benefits over Path Tracing AND Branched Path Tracing for various scenes.
I’m going to stop having this discussion now. Having Branched Path Tracing in Cycles-X would be nice. But it’s clear from the last year and 6 months that the Cycles developers do not believe Branched Path Tracing is the ideal solution to many problems ray tracing has, and there is a wide range of reasons why this might be the case. And so the Cycles developers are pursuing other techniques. And the work they have done is bringing large improvements to a wide range of scenes already.
These improvements may not be ideal for all situations, and Branched Path Tracing will still outperform new techniques in some areas. But overall I consider the switch to new techniques a overall positive, and if we have to sacrifice Branched Path Tracing to free up extra development time to speed up the development of the new techniques, then I’m happy with that. And I’m sure a large number of users probably feel the same.
The goal of Branched Path Tracing was to simply fire more rays per sample to overcome some of the short comings of ray tracing.
The goal of Cycles-X with techniques like Light Trees and Path Guiding is to use the small number of rays we typically have in a more efficient way by directing the rays in more efficient directions.
This trend of “lets use a small number of rays more efficiently” is a trend that’s happening in a lot of places in the ray tracing industry, and it started in the late 1900s with techniques like “Next Event Estimation”. And it has only speed up with time.
Especially with Nvidia and various game studios working hard on using rays more efficiently due to their investment in real time ray tracing for video games where only a small number of rays can actually be traced. And other companies with production ready render engines are also applying these sorts of techniques. So there must be some merit in this approach, either in terms of quality, performance, or both.
Yes, but new UI & UI bug fixes not exist in v293. Also, now I must be careful not to load v293.blend by v350 program and save it over…then cannot load it using v293 program.
Sorry, but give me a break here. Not like there is much to work here … just copy and paste “we dont fix it, its legacy stuff” as response. I had reported some bug and then discarded quickly by one sentence…and then I learned what LTS really is…it is not Long Term Support…it is Limited Time Support. All over the net this LTS bullshit (sorry) is marketing term like “we really care”…but really is “we dont give a shit, dont have any time to do it”. Please be truthful, no one is so powerful to fix everything…its understood, dont play branding as you can pull it like some god.
Play with BRV MRV SRV and experiment what ever you want, but, put BPT aside, freeze it, and leave it on the menu. That is I would do, give people some steady fixed standard available, and do experiments on new stuff as much you like.
Branching… separate engine and do not share libs anymore. Freeze old BPT libs. Cure only critical bugs in BPT. And, label it legacy.
This would eliminate 99% of maintenance and you will have your spare time to “progress” in any direction you want.
If Nvidia and others are really into Ray tracing bunch of polygons, they would put more than 24Gb into 4090 video card. U have actually NVIDIA RTX A6000 @ 48Gb (witch is better but still poor)
Now compare prices…
4090 @ 24Gb … 2K Euros
A6000 @ 48Gb … 5K Euros
It is obvious that games do not need more memory…and Nvida market is there. So, OPTIX and etc, is just marketing, to have some more for bragging…not real dedication. And, of course, we “ray tracers” are just small fragment of market…and what market needs gives response in terms of product quality/quantity/price.
I agree, they dont change existing scenes…but new scenes they are going to build will certainly be more complex to achieve more real results.
P.S. Look at my case…bought new proc…x3 boost…now I put more samples…from 100 to 150…add few more things on scene and finally calculating x2 boost at most. This is now full animation rendering of cca 3 min @ 30-60fps put 2 months to 1 month.
Of course, that is BPT…if I would use new progressive engine, I would be done rendering animation in few years.
And for me, buying 4090 does not help me coz scene is already at +20GB while rendering.
Buying A6000 is not an option. I dont give +3K Eur for +24Gb.
P.S.S. My calculation, for fine render of an outdoor scene is about 256Gb tag, and for achieving “full” reality is about 1T. U can do bumping, linking and etc to save memory but scene will look plastic and not convincing. To get somewhere near reality, you need tons of polygons/particles/large textures/.
If you are one frame coffee cup renderer, then use ZX Spectrum…it is enough.
Fascinating, I will investigate as soon LT is on multi-core.
Currently, tried and simply didnt help, same result as without LT…not sharp as BPT. I tell you, I stop counting how much more I need samples to get near BPT result… And, where is the catch I dont know…some other parameters are changed in engine…or something else…dont know.
Certainly, I have found case where BPT is master and blaster.
P.S. I use mesh lights…U didnt specify if scene is 12000 “standard light” lamp sources.
I’ve been following this thread a bit, however I’ve been unable to read every post.
What I see here is that there is some specific scene that seems to benefit from BPT, but at the same time it seems to be a corner case, so I don’t see a point on discussing it to the end of time.
One possibility could be to privately hire a developer to make an study and evaluate the cost of re-implementing BPT, but that comes with a big responsibility for mainteinance and compatibility, for example if BPT is re-introduced, then it should be compatible with Shadow Caustics and with Path Guiding, also with other features that are already here or coming soon, like LIght Groups and Light Linking, I think that’s not easy, and all those features would have been way more difficult and costly to implement including BPT (if they were even possible).
What I don’t see are specifics of the problematic scene to see if there is a possibility of helping, optimizing the scene, because in general as can be seen in Alaska’s tests, LT is way better, at least in that scene, than BPT.
In the end I don’t get the reason to continue with this thread, it’s a bit in loop, or it feels that way, BPT is gone, for many reasons, and won’t come back anytime soon, if there are corner cases that are way better with BPT than LT / PT in general, could be good to post them as demo scenes so devs can see what the problems are, what’s the big difference and what can be optimized, that would be helpful for the present and the future.
IMHO to advocate for BPT coming back is a bit of lost time, in general, the fact that BPT is gone has been way better for Cycles in general because it allows to get improvement and development at a faster pace.
About the “it depends on the scene”, every render engine feature performance depends on the scene, some scene benefits more from Path Guiding than other, some scene benefits more of limiting the amount of bounces than others, and of course some scenes benefits more from using LT than others, that’s nothing new.
¿In the future? who knows, as @Alaska said, it may come back, or at least part of it, but I imagine that it won’t happen until Cycles is on pair, or even in front of other state of the art render engines out there, we are way behind in many features required for production, and now we are getting them at a very good pace.
One last comment: please don’t see this as a request to close this thread, or wanting to silence anything about this, it’s just that I’m following it and in the end I think there are some things that are a lost cause and focusing efforts in other places could be more beneficial, but if you want to continue with this topic, please by no means understand that I want to silence anyone, just expressing my thoughts on the topic because i see it a lot in my inbox lately
Now, v36a…set time limit 5 min, start…building Light Tree…memory jump to 26Gb…then 28…finally hitting 32Gb limit…goes to virtual memory paging aka SSD is now memory surrogate…slows…but…survives!
Only 2 min for building LT…using paging. And now goes to render +5 min more.
Result…nowhere near to v293.17 BPT quality. Not just ocean is noisy, lost details, sharpness, even simple white flatly model in the center, has much more noise.
v293.17 BPT is obvious solution in my case…no extra memory price tag, no extra time to build LT…and…5-10 times faster render!?!
(While rendering using v293.17 BPT I was astound that central model had no obvious noise…I had to zoom it to notice noise.)
In that example you are mentioning I see several things:
Proc usage spikes in the new Cycles: yes, the new Cycles seems to not leverage, at least steadily, all the threads and/or devices like multiple GPUs, that’s somethihng on the works I think, but in general that’s compensated because in the majority of cases it’s way faster.
It does seem that it’s not your case.
Regarding the LT/BPT in your scene, the problem is that since we cannot see the scene we have no idea of what is going on there, you mention an ocean but we have no idea of how is the scene, maybe the problem is somewhere else and not in LT, things changed a lot between 2.9x and the new Cycles so you may be suffering some other type of penalty.
With that said I totally understand you being unable to show the actual scene, so the best way to proceed in this case is to upload a similar scene that suffers from the same problem but it’s not the actual production scene, that way Alaska or other developers could test it and see what is the problem, that could help to improve LT or could help detecting some other caveats.
(BTW I said before that I have not read all the messages, so it’s totally possible that you already shared the file and I missed it )
Regarding the alarm sounding XD : it’s not an alarm at all, it’s that I see the conversation going in circles to nowhere, mainly because BPT is not coming back, so I think it could be deviated towards something more productive like what I’ve said, or not, once again, totally your shot, I don’t want to silence anyone, if both of you like to keep going, I have no problem at all, but sometimes when a conversation is going in circles, someone from the outside making a ping could help inproving it, that’s it
P.S.: of course with blender the good thing is that you can always go back, like what you are doing, to overcome some problems
