GPU tests and benchmarks on buildbot

Technical Feedback
1

GPU Tests

@blender-bot build +gpu now builds GPU binaries and runs GPU tests on Linux + NVIDIA and macOS + Apple Silicon machines. This includes Cycles, EEVEE, Workbench, Hydra and the Compositor. See blender-bot docs.

There are various tests known to fail still, due to platform differences and some bugs. So a failing build here for now does not mean a pull request introduces a problem, the specific errors need to be checked. I hope we can get it passing in the near future.

Current results:

Benchmarks

There is a new nightly benchmarks builder that runs performance tests on the same two machines. This is mainly useful for Cycles at the moment, to detect unexpected performance regressions.

Improving and adding other performance tests is welcome. It’s a matter of making changes to the tests/performance scripts in the Blender repository.

Current results (with just a few data points so there is not much to see yet).

These results should be taken with a grain of salt, as there can be fluctuations for various reasons, or intentional changes that make quality vs. performance trade-offs.

4 Likes
2

Very interested in performance testing for geometry nodes!/ Also interested in diagram of differences for different files between main and pr.

3

Yes, it would be good to support the benchmarking for pull requests too. It’s a bit more involved since it requires building two Blender revisions and comparing them, or somehow comparing to a recent nightly git hash.

Remaining tasks are tracked here:

1 Like