This topic aims to gather some feedback on this GSoC project.
Here’s a link to the Proposal, and to the branch soc-2019-embree-gpu
Synopsis
Nobody likes waiting for hours whilst blender is busy rendering. Cycle is a heavy user of ray-tracing, and it require to build a BVH (Bounding Volume Hierarchy). The construction of such a tree is complicated. Currently blender use it’s own BVH Builder.
Currently, Embree can be used only if rendering on CPU, and it requires that an optional flag is set at compilation time (which is not enabled for pre-built binary).
The goal of this proposal is to make Embree also usable on GPU.
How to use ?
To enable this feature, Blender must be built with Embree (cmake flag WITH_CYCLES_EMBREE), then render with Cycle on a CUDA compatible GPU, and enable Performance > Acceleration Structure > Use Embree on GPU.
Current state: The branch compile and can render basic scene.
Currently there is no interest in this feature: BVH trees are less efficient than those produced by the original implementation. This is mainly due by an non-optimal SAH function that is used by Embree, and no instanced mesh handling.
Can we expect improvements in the pre-load times also? (other than render times)
And can be BVH and pre-load times improved with this initiative?
I would like to test trying to render 10 million particles in your build when you think is ready to be tested, because particle instances creation time is fast, but the load and BVH are slooooooow, so maybe this could improve with this
the BVH builder is part of embree (this gsoc will take that output and mash it into a usable format for the gpu), so if all you want to measure is the build time just grab any embree enabled build and try.
It depends of what you call pre-load times.
I count as render time : BVH building + Copy to Device + Rendering, so I guess that pre-load time is the 2 first steps.
Embree is using most of the CPU power by using optimized instruction (like AVX, SSE), so it should be able to improve the BVH Building Step
To improve render time, Embree support Spatial-Temporal BVH (STBVH, more detail https://www.embree.org/papers/2017-HPG-msmblur.pdf). It give better performance for scene with motion blur. Apart from that, the BVH Tree should have approximately the same quality.
Before testing with particles, I think that instantiating is a prerequisite, however it is not yet implemented.
Currently the SAH function does not take in account the number of primitive, the visibility flag, nor the time range. This is due to the usage of the very limited rtcBuildBVH function. To support fancy features of Embree, I’ll need to use the plain Embree API, that does not support exporting. There are way to access those data, but it use the internal Embree data structures (that might change between Embree updates, example : BVH Access ). Switching to this other API is planned for next week
Hi.
Do I need some kind of special setup to build this branch?
I get this error (Linux):
Scanning dependencies of target cycles_bvh
[ 23%] Building CXX object intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/bvh.cpp.o
[ 23%] Building CXX object intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/bvh2.cpp.o
[ 23%] Building CXX object intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/bvh4.cpp.o
[ 23%] Building CXX object intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/bvh8.cpp.o
[ 23%] Building CXX object intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/bvh_binning.cpp.o
[ 23%] Building CXX object intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/bvh_build.cpp.o
[ 23%] Building CXX object intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/bvh_embree.cpp.o
In file included from /media/disk1/@Blender_Git/blender/intern/cycles/bvh/embree/kernels/common/…/…/common/sys/…/math/vec2.h:19:0,
from /media/disk1/@Blender_Git/blender/intern/cycles/bvh/embree/kernels/common/…/…/common/sys/string.h:20,
from /media/disk1/@Blender_Git/blender/intern/cycles/bvh/embree/kernels/common/default.h:29,
from /media/disk1/@Blender_Git/blender/intern/cycles/bvh/embree/kernels/common/scene.h:19,
from /media/disk1/@Blender_Git/blender/intern/cycles/bvh/bvh_embree_converter.h:32,
from /media/disk1/@Blender_Git/blender/intern/cycles/bvh/bvh_embree.cpp:57:
/media/disk1/@Blender_Git/blender/intern/cycles/bvh/embree/kernels/common/…/…/common/sys/…/math/math.h:257:5: warning: “_WIN32” is not defined, evaluates to 0 [-Wundef] #if _WIN32
^~~~~~
In file included from /media/disk1/@Blender_Git/blender/intern/cycles/bvh/embree/kernels/common/scene.h:19:0,
from /media/disk1/@Blender_Git/blender/intern/cycles/bvh/bvh_embree_converter.h:32,
from /media/disk1/@Blender_Git/blender/intern/cycles/bvh/bvh_embree.cpp:57:
/media/disk1/@Blender_Git/blender/intern/cycles/bvh/embree/kernels/common/default.h:54:10: fatal error: …/config.h: No existe el archivo o el directorio #include “…/config.h”
^~~~~~~~~~~~~
compilation terminated.
intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/build.make:206: recipe for target ‘intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/bvh_embree.cpp.o’ failed
make[2]: *** [intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/bvh_embree.cpp.o] Error 1
CMakeFiles/Makefile2:1291: recipe for target ‘intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/all’ failed
make[1]: *** [intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/all] Error 2
Makefile:162: recipe for target ‘all’ failed
make: *** [all] Error 2
“No existe el archivo o el directorio” means “The file or folder does not exist.”
I can successfully build Master with Embree support in Linux.
It’s a branch, on top of that a gsoc student branch, things may not always be compiling and if they are they may not compile on all our supported platforms.
looks like that #if needs to be an #ifdef , i’d fix that first and see if the second error goes away.
Thank you. The first error disappeared, I still get the other error about “/media/disk1/@Blender_Git/blender/intern/cycles/bvh/embree/kernels/common/default.h:54:10: fatal error: …/config.h: ‘No existe el archivo o el directorio’” < ‘The file or folder does not exist’
Scanning dependencies of target cycles_bvh
[ 23%] Building CXX object intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/bvh.cpp.o
[ 23%] Building CXX object intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/bvh2.cpp.o
[ 23%] Building CXX object intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/bvh4.cpp.o
[ 23%] Building CXX object intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/bvh8.cpp.o
[ 23%] Building CXX object intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/bvh_binning.cpp.o
[ 23%] Building CXX object intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/bvh_build.cpp.o
[ 23%] Building CXX object intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/bvh_embree.cpp.o
In file included from /media/disk1/@Blender_Git/blender/intern/cycles/bvh/embree/kernels/common/scene.h:19:0,
from /media/disk1/@Blender_Git/blender/intern/cycles/bvh/bvh_embree_converter.h:32,
from /media/disk1/@Blender_Git/blender/intern/cycles/bvh/bvh_embree.cpp:57:
/media/disk1/@Blender_Git/blender/intern/cycles/bvh/embree/kernels/common/default.h:54:10: fatal error: …/config.h: No existe el archivo o el directorio #include “…/config.h”
^~~~~~~~~~~~~
compilation terminated.
intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/build.make:206: recipe for target ‘intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/bvh_embree.cpp.o’ failed
make[2]: *** [intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/bvh_embree.cpp.o] Error 1
CMakeFiles/Makefile2:1291: recipe for target ‘intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/all’ failed
make[1]: *** [intern/cycles/bvh/CMakeFiles/cycles_bvh.dir/all] Error 2
Makefile:162: recipe for target ‘all’ failed
make: *** [all] Error 2
By the way, I have Embree 3.2.4 installed in /opt/lib by install_deps.sh script.
It was my fault
Indeed the config.h is required, but it was in a .gitginore, so it was on my computer, but not in the git. And I didn’t detect this failure because this file isn’t required on Window (on which I double check that everything compile) …
Anyway, I added the file, so it should be better now
In my experience embree on CPU rendering is faster than the normal system, albeit it’s more inestable and it crashes, but it was faster, is Embree on GPU also faster than the normal system? (Without Motion Blur I mean)
With the current implementation, in scene with only a little bit of motion blur, the render time is similar: the performance improvement is lost in the conversion of the tree (from BVH4 to BVH2). But in the scene with deformation blur, the performance improvement is even higher, and far exceeds the losses.
On CPU it’s using BVH4 (and even BVH8 if possible) because on modern CPU, it’s possible to do 4 (or even 8) comparison at the same time, so traversing a BVH4 (or 8) can be done without cost on CPU. On the other hand, the GPU is not able to do that, so it will be 4 (8) times slower than a BVH2. But Embree being a CPU only library, it prefer to work with BVH4. So before sending the tree to the GPU, we need to convert this BVH4 to BVH2 (that step will only be done for GPU rendering), and during this step, the tree is losing quality.
I was not aware of such a feature. I quickly looked in the code, and the BVH tree is the same on all devices (CPU and GPU). The only tree kind compatible on both is BVH2. Thus using Embree or not wont change anything here.
, so it was on my computer, but not in the git. And I didn’t detect this failure because this file isn’t required on Window (on which I double check that everything compile) …