Embree for Cycles, cmake cannot find Embree

Building Blender
1

Hello all,

Compiling Blender is usually quite easy but I’m running into an issue when I enable Cycles/Embree in cmake options on MacOS (Mojave).
I’ve installed embree with brew so it’s under /usr/local/Cellar/embree/3.5.2

I tried :
cmake ../build_darwin/ -D_embree_LIBRARIES=/usr/local/Cellar/embree/3.5.2/lib

-- Could NOT find EMBREE (missing: _embree_LIBRARIES)
WARNING: Target "cycles" requests linking to directory "/usr/local/Cellar/embree/3.5.2/lib/".  Targets may link only to libraries.  CMake is dropping the item.

So I tried:
cmake ../build_darwin/ -DEMBREE_LIBRARIES=/usr/local/Cellar/embree/3.5.2 /lib/libembree3.dylib

-- Could NOT find EMBREE (missing: _embree_LIBRARIES)
-- Generating done

Embree is still not found and make fails. ‘embree3/rtcore.h’ file not found

Ultimately, I tried (to no avail):
cmake ../build_darwin/ -DEMBREE_LIBRARIES=/usr/local/Cellar/embree/3.5.2/lib/libembree3.dylib -DEMBREE_INCLUDE_DIR=/usr/local/Cellar/embree/3.5.2/include/embree3

Has anyone run into this issue before?

1 Like
2

Same issue here mate on Linux (Arch). Tried using the dependency script to build embree, with force-embree & build-embree & with-embree as option but it seems to skip it.
Installing embree with the package manager does also not yield any positive results.
Additionally i just got the following error from the install_dependencies.sh script:

image
image.png1215×95 25 KB

3

make deps is the most reliable way of building Embree for Cycles. Stock versions of Embree will not work correctly, as they typically don’t have all of the necessary features enabled.

1 Like
4

@StefanW this is the sort of magic command I don’t know. I tried it but after compiling a long time, external_boost failed with this error

...updated 14953 targets...
[ 17%] Performing install step for 'external_boost'
.
[ 18%] Completed 'external_boost'
[ 18%] Built target external_boost
make[1]: *** [all] Error 2
make: *** [deps] Error 2

I suppose you have no more magic command in your bag to solve this?

5

It would help if you could post the actual error from the log.

6

More information on make deps is here:
https://wiki.blender.org/wiki/Building_Blender/Dependencies
https://developer.blender.org/diffusion/B/browse/master/build_files/build_environment/CMakeLists.txt

7

make deps is on the chatty side of things, and multithreaded so generally when you hit the point where it says ‘error’ the actual error was probably 5 minutes ago.

Best thing to do when it errors out is run make single-threaded and watch for the actual error to occur

cd ../build_linux/deps
make -j1 install
1 Like
8

@LazyDodo thanks that really helps, I forgot about the multithreading and I could not find how to show more of an error log as @jesterKing requested.

So here’s the log:

[ 18%] Performing install step for 'external_opencolorio'
[  5%] Built target YAML_CPP_LOCAL
[ 11%] Built target tinyxml
[ 55%] Built target OpenColorIO_STATIC
[100%] Built target ocio_core_tests
Install the project...
-- Install configuration: "Release"
-- Installing: /some_path/repositories/blender_28_mod/build_darwin/deps/Release/opencolorio/share/ocio/setup_ocio.sh
-- Installing: /some_path/repositories/blender_28_mod/build_darwin/deps/Release/opencolorio/cmake/OpenColorIO.cmake
-- Installing: /some_path/repositories/blender_28_mod/build_darwin/deps/Release/opencolorio/cmake/OpenColorIO-release.cmake
-- Installing: /some_path/repositories/blender_28_mod/build_darwin/deps/Release/opencolorio/./OpenColorIOConfig.cmake
-- Installing: /some_path/repositories/blender_28_mod/build_darwin/deps/Release/opencolorio/lib/static/libOpenColorIO.a
-- Installing: /some_path/repositories/blender_28_mod/build_darwin/deps/Release/opencolorio/include/OpenColorIO/OpenColorIO.h
-- Installing: /some_path/repositories/blender_28_mod/build_darwin/deps/Release/opencolorio/include/OpenColorIO/OpenColorTransforms.h
-- Installing: /some_path/repositories/blender_28_mod/build_darwin/deps/Release/opencolorio/include/OpenColorIO/OpenColorTypes.h
-- Installing: /some_path/repositories/blender_28_mod/build_darwin/deps/Release/opencolorio/include/OpenColorIO/OpenColorABI.h
-- Installing: /some_path/repositories/blender_28_mod/build_darwin/deps/Release/opencolorio/lib/pkgconfig/OpenColorIO.pc
[ 18%] Completed 'external_opencolorio'
[ 18%] Built target external_opencolorio
[ 20%] Built target external_lame
[ 20%] Performing build step for 'external_vpx'
    [AS] vpx_ports/emms.asm.o
nasm: error: unable to find utility "nasm", not a developer tool or in PATH
make[4]: *** [vpx_ports/emms.asm.o] Error 72
make[3]: *** [.DEFAULT] Error 2
make[2]: *** [build/vpx/src/external_vpx-stamp/external_vpx-build] Error 2
make[1]: *** [CMakeFiles/external_vpx.dir/all] Error 2
make: *** [all] Error 2
9

There’s a bunch of build time dependencies that you need, these include (from the top of my head, i may or may not have missed one here) yasm , nasm (>=2.12), tcl , flex , bison , automake and autoconf

10

The necessary tools are mentioned briefly here:

https://developer.blender.org/diffusion/B/browse/master/build_files/build_environment/CMakeLists.txt

The relevant line:

Install with homebrew: brew install cmake autoconf automake libtool yasm nasm

11

Thanks, @LazyDodo for pointing me here, and I hope not to hijack this thread, but I’m facing the exact same issue with openSUSE Tumbleweed builds, available here:
https://build.opensuse.org/project/monitor/home:frispete:blender

In the hope to have created embree to Blenders likings:

cmake \
    -DCMAKE_C_FLAGS:STRING="$CFLAGS %{optflags} -fPIC ${sseflags}" \
    -DCMAKE_CXX_FLAGS:STRING="$CXXFLAGS %{optflags} -fPIC ${sseflags}" \
    -DCMAKE_BUILD_TYPE=RelWithDebInfo \
    -DEMBREE_LIB_INSTALL_DIR=%{_libdir} \
    -DEMBREE_ISPC_SUPPORT=OFF \
    -DEMBREE_TUTORIALS=OFF \
    -DEMBREE_RAY_MASK=ON \
    -DEMBREE_FILTER_FUNCTION=ON \
    -DEMBREE_BACKFACE_CULLING=OFF \
    -DEMBREE_TASKING_SYSTEM=INTERNAL \
    -DEMBREE_MAX_ISA=AVX2

Blender isn’t able to use this build:

[  300s] -- Could NOT find EMBREE (missing: _embree_LIBRARIES)

I’ve tried a lot of combinations, but failed so far:

cmake [...]
      -DWITH_CYCLES_EMBREE:BOOL=ON \
      -DEMBREE_INCLUDE_DIR=/usr/lib64 \
      -DEMBREE_ROOT_DIR=/usr \

Embree uses the typical lib layout:

lrwxrwxrwx    1 root    root                       19 Jul 17 13:20 /usr/lib64/libembree3.so.3 -> libembree3.so.3.5.2
-rwxr-xr-x    1 root    root                 26589728 Jul 17 13:22 /usr/lib64/libembree3.so.3.5.2

as well as does the devel package:

drwxr-xr-x    2 root    root                        0 Jul 17 13:20 /usr/include/embree3
-rw-r--r--    1 root    root                     1416 Mär 13 12:13 /usr/include/embree3/rtcore.h
-rw-r--r--    1 root    root                     1443 Mär 13 12:13 /usr/include/embree3/rtcore.isph
-rw-r--r--    1 root    root                     2541 Mär 13 12:13 /usr/include/embree3/rtcore_buffer.h
-rw-r--r--    1 root    root                     2616 Mär 13 12:13 /usr/include/embree3/rtcore_buffer.isph
-rw-r--r--    1 root    root                     4690 Mär 13 12:13 /usr/include/embree3/rtcore_builder.h
-rw-r--r--    1 root    root                     6342 Mär 13 12:13 /usr/include/embree3/rtcore_common.h
-rw-r--r--    1 root    root                     6103 Mär 13 12:13 /usr/include/embree3/rtcore_common.isph
-rw-r--r--    1 root    root                     3928 Mär 13 12:13 /usr/include/embree3/rtcore_device.h
-rw-r--r--    1 root    root                     4029 Mär 13 12:13 /usr/include/embree3/rtcore_device.isph
-rw-r--r--    1 root    root                    14123 Mär 13 12:13 /usr/include/embree3/rtcore_geometry.h
-rw-r--r--    1 root    root                    16754 Mär 13 12:13 /usr/include/embree3/rtcore_geometry.isph
-rw-r--r--    1 root    root                    11825 Mär 13 12:13 /usr/include/embree3/rtcore_ray.h
-rw-r--r--    1 root    root                    10893 Mär 13 12:13 /usr/include/embree3/rtcore_ray.isph
-rw-r--r--    1 root    root                     6881 Mär 13 12:13 /usr/include/embree3/rtcore_scene.h
-rw-r--r--    1 root    root                     9094 Mär 13 12:13 /usr/include/embree3/rtcore_scene.isph
-rw-r--r--    1 root    root                     2469 Mär 13 12:13 /usr/include/embree3/rtcore_version.h
drwxr-xr-x    2 root    root                        0 Jul 17 13:20 /usr/lib64/cmake/embree3
-rw-r--r--    1 root    root                     1504 Jul 17 12:50 /usr/lib64/cmake/embree3/embree-config-version.cmake
-rw-r--r--    1 root    root                     2596 Jul 17 12:50 /usr/lib64/cmake/embree3/embree-config.cmake
lrwxrwxrwx    1 root    root                       15 Jul 17 13:20 /usr/lib64/libembree3.so -> libembree3.so.3

The full blender build log is available here: https://build.opensuse.org/package/live_build_log/home:frispete:blender/blender-git/openSUSE_Tumbleweed/x86_64.

And the packages are located here (cannot paste more than 2 links, sorry for the fuzz):

Embree: https://build.opensuse.org/package/show/home:frispete:blender/embree
Blender: https://build.opensuse.org/package/show/home:frispete:blender/blender-git
12

You’re missing -DEMBREE_STATIC_LIB=ON.
It’s there for a reason:
Embree as shared library is just a single libembree.so. As a static lib on the other hand, it consists of several .a files that need to be linked in the correct order. Since official Blender builds use static linking, the build process is set up for the latter case.

1 Like
13

You’re missing -DEMBREE_STATIC_LIB=ON

Just as I had feared… Distribution package builders hate static libs like the devil avoids holy waters…

Please allow me the question, what’s the rational of binding it statically, other than raising complexity on linking (order)?

14

The official builds are intended to be used across many different distributions and to run out of the box, and static linking is the best way to ensure that every dependency is fulfilled.

With regards to Embree directly, there are many different ways in which Embree can be built (different ray tracing features enabled/disabled, ISPC support, packet traversal, etc) and the out-of-the-box configuration of Embree does not work with Cycles. Unless a Linux distribution has a system to not only manage different versions of the same library but also different build configurations, I only foresee trouble when multiple applications rely on the Embree shared library, but expect different build configurations.

15

Okay, fair enough, and yes, in this light, the static linking is probably the way of least trouble.

Thanks for elaborating, @StefanW .

16

Just to be clear, I have nothing against dynamic linking. If I had seen an easy way to set this up to build against both dynamic and static Embree libraries, I would have done so. If someone finds a safe way of doing so, patches are welcome!

17

@LazyDodo and @jesterKing, thanks for the help but I got another error after hours of compiling.

[ 91%] Linking CXX executable iconvert
ld: warning: object file (/usr/local/lib/libwebp.a(libwebpencode_la-config_enc.o)) was built for newer OSX version (10.14) than being linked (10.11)
ld: warning: object file (/usr/local/lib/libwebp.a(libwebpdecode_la-webp_dec.o)) was built for newer OSX version (10.14) than being linked (10.11)
ld: warning: object file (/usr/local/lib/libwebp.a(libwebpencode_la-webp_enc.o)) was built for newer OSX version (10.14) than being linked (10.11)
ld: warning: object file (/usr/local/lib/libwebp.a(libwebpdecode_la-vp8_dec.o)) was built for newer OSX version (10.14) than being linked (10.11)

// skipping lots of ld:warning messages for readability sake

ld: warning: object file (/usr/local/lib/libopenjp2.a(t2.c.o)) was built for newer OSX version (10.14) than being linked (10.11)
ld: warning: object file (/usr/local/lib/libopenjp2.a(tgt.c.o)) was built for newer OSX version (10.14) than being linked (10.11)
ld: warning: object file (/usr/local/lib/libwebp.a(libwebpdsp_sse2_la-ssim_sse2.o)) was built for newer OSX version (10.14) than being linked (10.11)
ld: warning: object file (/usr/local/lib/libopenjp2.a(bio.c.o)) was built for newer OSX version (10.14) than being linked (10.11)
ld: warning: object file (/usr/local/lib/libopenjp2.a(mqc.c.o)) was built for newer OSX version (10.14) than being linked (10.11)
Undefined symbols for architecture x86_64:
  "boost::filesystem::detail::recur_dir_itr_imp::increment(boost::system::error_code*)", referenced from:
      OpenImageIO_v1_8::Filesystem::get_directory_entries(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >&, bool, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) in libOpenImageIO.a(filesystem.cpp.o)
  "boost::detail::set_tss_data(void const*, void (*)(void (*)(void*), void*), void (*)(void*), void*, bool)", referenced from:
      OpenImageIO_v1_8::pvt::ImageCacheImpl::ImageCacheImpl() in libOpenImageIO.a(imagecache.cpp.o)
      boost::thread_specific_ptr<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >::~thread_specific_ptr() in libOpenImageIO.a(imagecache.cpp.o)
      boost::thread_specific_ptr<OpenImageIO_v1_8::pvt::ImageCachePerThreadInfo>::~thread_specific_ptr() in libOpenImageIO.a(imagecache.cpp.o)
      OpenImageIO_v1_8::pvt::ImageCacheImpl::~ImageCacheImpl() in libOpenImageIO.a(imagecache.cpp.o)
      OpenImageIO_v1_8::pvt::ImageCacheImpl::get_perthread_info(OpenImageIO_v1_8::pvt::ImageCachePerThreadInfo*) in libOpenImageIO.a(imagecache.cpp.o)
      OpenImageIO_v1_8::pvt::ImageCacheImpl::append_error(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) const in libOpenImageIO.a(imagecache.cpp.o)
      boost::thread_specific_ptr<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >::~thread_specific_ptr() in libOpenImageIO.a(imageio.cpp.o)
      ...
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[5]: *** [src/iconvert/iconvert] Error 1
make[4]: *** [src/iconvert/CMakeFiles/iconvert.dir/all] Error 2
make[3]: *** [all] Error 2
make[2]: *** [build/openimageio/src/external_openimageio-stamp/external_openimageio-build] Error 2
make[1]: *** [CMakeFiles/external_openimageio.dir/all] Error 2
make: *** [all] Error 2

What should I do now? How to compile openimageio for Mojave (10.14) in this case?

18

@nantille. Maybe @akitula can give some more info - IIRC he has been working on compiling Blender on MacOS.

I would assume the make deps command would work also on Mojave, but tbh I am not sure on what version official Blender is built.

19

You seem to be using libraries from homebrew or some other package that are built for targeting 10.14. Make deps should work there, it will target 10.11 as the mmac-osx-version-min compiler flag needs to be set.

20

@jesterKing Thanks for pointing me to @akitula.

make deps yields the same error as above. I tried brew uninstall openimageio to see if it would then download and compile the version it needs but I still get the same error.

Btw, uninstalling from brew yields:
Uninstalling /usr/local/Cellar/openimageio/2.0.7_2

But make deps wants to use OIIO 1.8 so it doesn’t seem related.