honestly the benchmark couldn’t be any more difficult to use, especially with version 3 breaking the gui launcher and removing gpu support, while also removing support for the old launchers. Seriously I was trying to benchmark a new configuration today and now I simply cant. Im honestly just trying to quantify my systems performance and Id like to benchmark blender as its a legitimate piece of software but wow, I can run my own benchmarks in programs I actually use faster than having to learn how to arbitrarily download benchmarks and all of the arguments to make them run right.
Its actually kinda hilarious to me that the benchmark actually downloads the scenes every single time too, I really don’t understand why you guys like throwing away a gb of bandwidth from your servers any time someone technically uninclined wants to run the benchmark. Seriously this is a clownshow for benchmarking. Anyone actually interested in blender performance is much better off downloading the entire programing and benchmarking through it.
My name is Rob. I am a senior Business Intelligence consultant, located in the Netherlands with dabbling around in Blender as a hobby. I was looking for some open data, to simulate some kind of mini datawarehouse for our junior consultants to learn, administer and analyse a situation that is simular to the real world. The advantage of this data is that it is real, moving data and we can add our own infomation (our own benchmarks). For now I have analysed the data a bit and set up a process that downloads the zip file every day, to load it on our server as one big dataset. So far I made the flow in a program called SAS Viya, but one of my collegues is going to build the same in Microsoft Power BI. As all the data in open source and easy to access, I wonder if the conclusions of our analyses are interesting to share publicly. On the Blender Open Data page are some graph’s with the fastest GPU’s and CPU’s, but I am sure there are many meaningfull graphs to be made and to be shared. Also a better discription of the data like a data model we could provide to add to the About-page for example.
Is there any interest in this? If there is interest, how can we help you making this benchmark tool as effective a possible?
For now I have one question about the data that is not really clear to me: How are submissions with more than one GPU can be identified in the data?
Im not into analyzing data so I dont really now what to expect. What else can you derive from the data set?
The user is simply not allowed to benchmark multi GPU.
What GPU is actually picked out of a multi GPU setup is also not clear to me.
The Display GPU would be naturally a bit slower than the second, unconnected one.
Outliers in the data for example, just like the unrealistic values of the Apple M1 with Metal on the junkshop scene at the moment. A more in depth study of cpu’s en gpu’s when rendering Blender. Differences between scenes and Blender versions et cetera. There are a lot of possibilities for this data.
Hello i am looking for a way to get the split values of my Benchmark, at time i only took note of overall value. Any way to get it?
I found the the daily sets less than helpful.
Hi,
I’m interested in the enhancement of Blender with MetalRT (as opposed to Metal compute) over time. Could this be added to Benchmark as an API option?
Are you sure it actually is? I see very little performance improvement (+20%) over the previous benchmarks. The individual scene renders are showing a large improvement but not the benchmark.
I’m pretty sure it is enabled, 20% doesn’t sound wrong to me and where else could that come from?
If half of the time is spent in shading and half in ray-tracing, there can be a 50% render time reduction at most. But hardware ray-tracing is not free either, and I think it’s not done in hardware to the same extent as NVIDIA cards. There may be room to better take advantage of this in our code, it’s unknown still.
The dynamic caching and other architecture improvements probably have a bigger impact on performance, and that applies to 3.6 as well.
I’m seeing 50-100% improvements in the individual scene renders (at least Classroom & Junkyard) so the Benchmark performance doesn’t make sense. Something’s off - any idea where I can get info, is this forum still monitored?
Yes to all (except Monster). Blender 4.0 on M3 Max with Metal RT on vs M2 Max. Improvements are as you stated - 100% = 2x speed. Admittedly just scrapped from YouTube videos as I don’t have the test equipment but the same improvement across over half a dozen ‘reputable’ channels (as reputable as they can be). Just odd the performance increase is not reflected in the Benchmark.
I guess so, though I noticed the M2 Max scores have reduced by 9% between 3.6 & 4.0 which, coupled with the M3 Max 14% increase, shows a better ‘improvement’ just not the kind of improvement I was expecting.
The device gfx906 doesn’t have HIP binaries compiled for Cycles. And the Radeon VII and Radeon Pro VII are both gfx906 devices.
The fact there is a Radeon Pro VII result on Blender 4.0 is a bit odd. Someone could be using a custom build to achieve this (It’s possible to use custom versions of Blender in the OpenData benchmark if you time things right). HSA overrides may work to get this GPU working using the kernels for 1st gen Vega, but I don’t know enough about this to know if this even works in this case.
i hope this will read someone who has something to do with “this”
i think it is very “strange” that in “our” benchmark 4080 is faster than “4080 super”
The results you’re looking at may be influenced by previous Blender versions. If I go to the OpenData website and search for a RTX 4080 and RTX 4080 Super, and limit results to Blender 4.0.0 only, then the performance numbers are much closer (but with the RTX 4080 still being faster).
The reason this is having a impact is because some older versions of Blender/Cycles are faster than Blender 4.0.0. So if RTX 4080 test are distributed across Blender 3.6 and 4.0, while the RTX 4080 Super only has tests in Blender 4.0.0, then that will skew results towards the RTX 4080 “appearing faster”.
As for why the RTX 4080 is faster than the RTX 4080 Super on OpenData even when you filter to specific Blender versions, I don’t know.
