You can try removing the ../build_linux/deps/build/sqlite folder and building that library again from the start. It’s hard to tell anything from that snippet, would need to see the complete log, the actual problem is probably even earlier.
Hi brecht.
If “blender” folder is inside other folder whose name contains spaces, then “make deps” does not work. For example if the path is “/Disk/Blender Git/blender” and I run “make deps” inside “blender” folder, it gives an error about “/Disk/Blender Git/blender/Git/lib/linux_x86_64” does not exist. If I rename and remove the space, for example “/Disk/Blender_Git/blender”, it works. Could this be corrected to support folder names with spaces?
I removed the sqlite folder, as you suggested, and ran make again. These are the errors:
[ 78%] Performing download step (download, verify and extract) for 'external_sqlite'
-- Downloading...
dst='/home/haig/blender-git/build_linux/deps/build/sqlite/src/sqlite-src-3240000.zip'
timeout='none'
-- Using src='https://www.sqlite.org/2018/sqlite-src-3240000.zip'
-- verifying file...
file='/home/haig/blender-git/build_linux/deps/build/sqlite/src/sqlite-src-3240000.zip'
-- Downloading... done
-- extracting...
src='/home/haig/blender-git/build_linux/deps/build/sqlite/src/sqlite-src-3240000.zip'
dst='/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite'
-- extracting... [tar xfz]
-- extracting... [analysis]
-- extracting... [rename]
-- extracting... [clean up]
-- extracting... done
[ 78%] No patch step for 'external_sqlite'
[ 78%] No update step for 'external_sqlite'
[ 78%] Performing configure step for 'external_sqlite'
.
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for ar... ar
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
checking for a BSD-compatible install... /usr/bin/install -c
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... no
checking for int8_t... yes
checking for int16_t... yes
checking for int32_t... yes
checking for int64_t... yes
checking for intptr_t... yes
checking for uint8_t... yes
checking for uint16_t... yes
checking for uint32_t... yes
checking for uint64_t... yes
checking for uintptr_t... yes
checking for sys/types.h... (cached) yes
checking for stdlib.h... (cached) yes
checking for stdint.h... (cached) yes
checking for inttypes.h... (cached) yes
checking malloc.h usability... yes
checking malloc.h presence... yes
checking for malloc.h... yes
checking for fdatasync... yes
checking for gmtime_r... yes
checking for isnan... yes
checking for localtime_r... yes
checking for localtime_s... no
checking for malloc_usable_size... yes
checking for strchrnul... yes
checking for usleep... yes
checking for utime... yes
checking for pread... yes
checking for pread64... yes
checking for pwrite... yes
checking for pwrite64... yes
checking for tclsh8.7... no
checking for tclsh8.6... no
checking for tclsh8.5... no
checking for tclsh... no
Warning: can't find tclsh - defaulting to non-amalgamation build.
./configure: line 7595: tclsh: command not found
configure: Version set to 3.24
configure: Release set to 3.24.0
configure: Version number set to 3024000
checking whether to support threadsafe operation... yes
checking for library containing pthread_create... -lpthread
checking for library containing pthread_mutexattr_init... none required
checking whether to support shared library linked as release mode or not... no
checking whether to use an in-ram database for temporary tables... no
checking if executables have the .exe suffix... unknown
checking for Tcl configuration... ./configure: line 7975: tclsh: command not found
./configure: line 7989: tclsh: command not found
configure: WARNING: Can't find Tcl configuration definitions
configure: WARNING: *** Without Tcl the regression tests cannot be executed ***
configure: WARNING: *** Consider using --with-tcl=... to define location of Tcl ***
checking for library containing readline... -ledit
checking for library containing fdatasync... none required
checking zlib.h usability... yes
checking zlib.h presence... yes
checking for zlib.h... yes
checking for library containing deflate... -lz
checking for library containing dlopen... -ldl
checking whether to support MEMSYS5... no
checking whether to support MEMSYS3... no
checking for library containing log... -lm
checking for library containing log... (cached) -lm
configure: creating ./config.status
config.status: creating Makefile
config.status: creating sqlite3.pc
config.status: creating config.h
config.status: executing libtool commands
[ 79%] Performing build step for 'external_sqlite'
/bin/sh: tclsh: command not found
make[3]: *** [Makefile:1003: sqlite3.h] Error 127
make[2]: *** [CMakeFiles/external_sqlite.dir/build.make:115: build/sqlite/src/external_sqlite-stamp/external_sqlite-build] Error 2
make[1]: *** [CMakeFiles/Makefile2:1360: CMakeFiles/external_sqlite.dir/all] Error 2
make: *** [Makefile:130: all] Error 2There’s that warning about tcl, but I don’t really know what that means.
So, I installed tcl, removed sqlite folder, and ran make again. Now, I get:
Libraries have been installed in:
/home/haig/blender-git/build_linux/deps/Release/sqlite/lib
If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
- add LIBDIR to the `LD_LIBRARY_PATH' environment variable
during execution
- add LIBDIR to the `LD_RUN_PATH' environment variable
during linking
- use the `-Wl,-rpath -Wl,LIBDIR' linker flag
- have your system administrator add LIBDIR to `/etc/ld.so.conf'
See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
/usr/bin/install: cannot change permissions of ‘/usr/lib/tcl8.6/sqlite3’: No such file or directory
make[3]: *** [Makefile:1350: tcl_install] Error 1
make[2]: *** [CMakeFiles/external_sqlite.dir/build.make:74: build/sqlite/src/external_sqlite-stamp/external_sqlite-install] Error 2
make[1]: *** [CMakeFiles/Makefile2:1360: CMakeFiles/external_sqlite.dir/all] Error 2
make: *** [Makefile:130: all] Error 2Edit: usr/lib/tcl8.6/sqlite3 doesn’t exist. Hm… does a lib need to be linked? Env var set to something, maybe?
You can run the NPROCS=1 make for the GNUMakefile. It would be better if it understood -j and passed it, instead of reinventing the wheel, of course.
Actually, running make from inside ../build_linux/deps takes the -j1 option, I discovered with all your help.
I think you need to apt install tcl. Never heard of this being needed to build our libraries, but appears sqlite needs it.
Yes. I did install it, as I mentioned, and get the second set of errors (first set are gone after tcl) that I posted because /usr/lib/tcl8.6/sqlite3 isn’t present. I searched the system and couldn’t find sqlite3 anywhere, except in a python folder (that can’t be it).
So, what I did was (and this is on Arch, mind you) sudo mkdir /usr/lib/tcl8.6/sqlite3 and changed its ownership to user. Ran make again and voila! sqlite is built.
Dependencies successfully built and installed to /home/haig/blender-git/lib/linux_x86_64.YAY!
Now that they’re built, can I just go into my build folder, and run make from there? What do I have to do to my CMakeCache.txt besides set WITH_STATIC_LIBS=ON? I’m almost there - I can taste it…
If you do a completely clean build with a new CMakeCache.txt, you shouldn’t have to configure anything. It will automatically find that lib directory and enable static libraries.
Okay, so I hit a snag (I expected this). Everything was going great, until the end, which resulted in lib errors pretty much across OpenVDB and ffmpeg. I’m wondering if I shouldn’t have passed the cmake commands differently. @brecht you tell me. Here’s what I did:
- Entered the build directory
../sculpt-mode-features - Ran
cmake -C../blender/build_files/cmake/config/blender_full.cmake ../blender - Ran
cmake -U *SNDFILE* -U *PYTHON* -U *BOOST* -U *Boost* -U *OPENCOLORIO* -U *OPENEXR* -U *OPENIMAGEIO* -U *LLVM* -U *CYCLES* -U *OPENSUBDIV* -U *OPENVDB* -U *COLLADA* -U *FFMPEG* -U *ALEMBIC* -D WITH_CODEC_SNDFILE=ON -D PYTHON_VERSION=3.7 -D WITH_OPENCOLORIO=ON -D WITH_LLVM=ON -D LLVM_VERSION=6.0.1 -D LLVM_ROOT_DIR=/opt/lib/llvm -D LLVM_STATIC=ON -D WITH_OPENSUBDIV=ON -D WITH_OPENVDB=ON -D WITH_OPENVDB_BLOSC=ON -D WITH_ALEMBIC=ON -D ALEMBIC_ROOT_DIR=/opt/lib/alembic -D WITH_CODEC_FFMPEG=ON -D FFMPEG_LIBRARIES='avformat;avcodec;avutil;avdevice;swscale;swresample;lzma;rt;theora;theoradec;theoraenc;vorbis;vorbisenc;vorbisfile;ogg;x264;openjp2' -D WITH_OPENVDB_3_ABI_COMPATIBLE=ON -D WITH_CYCLES_CUDA_BINARIES=ON -D CUDA_SDK_ROOT_DIR=/opt/cuda/samples -D CUDA_TOOLKIT_ROOT_DIR=/opt/cuda/ .
This contains the recommended setting from the resultantBUILD_NOTES.txt, after runninginstall_deps.shway back when I was just doing regular builds. I added the OpenVDB and CUDA lines in order to make them compile properly and be included as static. - Ran
make -j12
Are libs clashing? How do you guys do it at the Institute?
Here’s a sample of the errors:
../../lib/libbf_intern_openvdb.a(openvdb_capi.cc.o):openvdb_capi.cc:function openvdb::v5_1abi3::Grid<openvdb::v5_1abi3::tree::Tree<openvdb::v5_1abi3::tree::RootNode<openvdb::v5_1abi3::tree::InternalNode<openvdb::v5_1abi3::tree::InternalNode<openvdb::v5_1abi3::tree::LeafNode<float, 3u>, 4u>, 5u> > > >::readTopology(std::istream&): error: undefined reference to 'openvdb::v5_1abi3::GridBase::saveFloatAsHalf() const'
../../lib/libbf_intern_openvdb.a(openvdb_capi.cc.o):openvdb_capi.cc:function openvdb::v5_1abi3::Grid<openvdb::v5_1abi3::tree::Tree<openvdb::v5_1abi3::tree::RootNode<openvdb::v5_1abi3::tree::InternalNode<openvdb::v5_1abi3::tree::InternalNode<openvdb::v5_1abi3::tree::LeafNode<float, 3u>, 4u>, 5u> > > >::writeTopology(std::ostream&) const: error: undefined reference to 'openvdb::v5_1abi3::GridBase::saveFloatAsHalf() const'
/home/haig/blender-git/blender/../lib/linux_x86_64/ffmpeg/lib/libavcodec.a(libxvid_rc.o):libxvid_rc.c:function ff_xvid_rate_estimate_qscale: error: undefined reference to 'xvid_plugin_2pass2'
/home/haig/blender-git/blender/../lib/linux_x86_64/ffmpeg/lib/libavcodec.a(libxvid_rc.o):libxvid_rc.c:function ff_xvid_rate_estimate_qscale: error: undefined reference to 'xvid_plugin_2pass2'
/home/haig/blender-git/blender/../lib/linux_x86_64/ffmpeg/lib/libavcodec.a(libxvid_rc.o):libxvid_rc.c:function ff_xvid_rate_control_init: error: undefined reference to 'xvid_plugin_2pass2'There’s a lot in between, but you get the idea. All has to do with libs.
You should not do any CMake library configuration at all, like this:
cd ../sculpt-mode-features
rm CMakeCache.txt
cmake -C../blender/build_files/cmake/config/blender_full.cmake ../blender
make -j12
Very interesting. ALMOST THERE! With your method, previous errors are gone, but now I get all OpenImageIO errors:
/home/haig/blender-git/lib/linux_x86_64/openimageio/lib/libOpenImageIO.a(webpinput.cpp.o):webpinput.cpp:function OpenImageIO_v1_8::webp_imageio_library_version(): error: undefined reference to 'WebPGetDecoderVersion'
/home/haig/blender-git/lib/linux_x86_64/openimageio/lib/libOpenImageIO.a(webpinput.cpp.o):webpinput.cpp:function OpenImageIO_v1_8::webp_pvt::WebpInput::open(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, OpenImageIO_v1_8::ImageSpec&): error: undefined reference to 'WebPGetInfo'
/home/haig/blender-git/lib/linux_x86_64/openimageio/lib/libOpenImageIO.a(webpinput.cpp.o):webpinput.cpp:function OpenImageIO_v1_8::webp_pvt::WebpInput::open(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, OpenImageIO_v1_8::ImageSpec&): error: undefined reference to 'WebPDecodeRGBA'
/home/haig/blender-git/lib/linux_x86_64/openimageio/lib/libOpenImageIO.a(webpoutput.cpp.o):webpoutput.cpp:function OpenImageIO_v1_8::webp_pvt::WebpOutput::close(): error: undefined reference to 'WebPPictureFree'
/home/haig/blender-git/lib/linux_x86_64/openimageio/lib/libOpenImageIO.a(webpoutput.cpp.o):webpoutput.cpp:function OpenImageIO_v1_8::webp_pvt::WebpOutput::write_scanline(int, int, OpenImageIO_v1_8::TypeDesc, void const*, long): error: undefined reference to 'WebPPictureImportRGB'
/home/haig/blender-git/lib/linux_x86_64/openimageio/lib/libOpenImageIO.a(webpoutput.cpp.o):webpoutput.cpp:function OpenImageIO_v1_8::webp_pvt::WebpOutput::write_scanline(int, int, OpenImageIO_v1_8::TypeDesc, void const*, long): error: undefined reference to 'WebPEncode'
/home/haig/blender-git/lib/linux_x86_64/openimageio/lib/libOpenImageIO.a(webpoutput.cpp.o):webpoutput.cpp:function OpenImageIO_v1_8::webp_pvt::WebpOutput::write_scanline(int, int, OpenImageIO_v1_8::TypeDesc, void const*, long): error: undefined reference to 'WebPPictureImportRGBA'
/home/haig/blender-git/lib/linux_x86_64/openimageio/lib/libOpenImageIO.a(webpoutput.cpp.o):webpoutput.cpp:function OpenImageIO_v1_8::webp_pvt::WebpOutput::open(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, OpenImageIO_v1_8::ImageSpec const&, OpenImageIO_v1_8::ImageOutput::OpenMode) [clone .part.141]: error: undefined reference to 'WebPPictureInitInternal'
/home/haig/blender-git/lib/linux_x86_64/openimageio/lib/libOpenImageIO.a(webpoutput.cpp.o):webpoutput.cpp:function OpenImageIO_v1_8::webp_pvt::WebpOutput::open(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, OpenImageIO_v1_8::ImageSpec const&, OpenImageIO_v1_8::ImageOutput::OpenMode) [clone .part.141]: error: undefined reference to 'WebPConfigInitInternal'
collect2: error: ld returned 1 exit status
make[2]: *** [source/creator/CMakeFiles/blender.dir/build.make:586: bin/blender] Error 1
make[1]: *** [CMakeFiles/Makefile2:7908: source/creator/CMakeFiles/blender.dir/all] Error 2
make: *** [Makefile:163: all] Error 2What am I missing here?
I’ve run into similar issues recently. Try installing OpenImageIO from your package manager, there’s a chance it’s looking for it in your system libraries.
If the purpose is to make a static portable build, installing OIIO or any other dependencies through the package manager is more likely to break things. You do not want Blender to use those.
The problem here is likely that the OIIO build as part of make deps picked up the WebP library installed on the system. We tell OIIO not to use it, but there seems to be a missing option for that in the OIIO cmake files.
The solution would be to edit ../build_linux/deps/CMakeCache.tx and turn on WITH_WEBP, then build the dependencies again.
I deleted OpenImageIO from the ../build_linux/deps/build folder, turned on WITH_WEBP in CMakeCache.txt, and ran make again. Then, rebuilt blender. Stops at the end with different errors, but still with OIIO. Blasted the entire ../build_linux/deps/build folder and am currently rebuilding all the libs. We’ll see where that leads.
Rebuilt all libs. Started fresh with the branch build folder ../sculpt-mode-features. Rebuilt Blender. Worked. Everything seems to be there.
One thing: I’ve never built anyone else’s branch (only ever my own, for my own purposes); and I’ve never built a completely static package. This is the first. I noticed in the buildbot versions, there’s only libGL.so and 'libGLU.solibs the…/sculpt-mode-features/libfolder. Whereas, in mine, there's a plethora, which is fine - BUT, nolibGL.soorlibGLU.so`.
Now, in order to get CUDA to recognize my CUDA devices, I had to add the:
cmake -D WITH_CYCLES_CUDA_BINARIES=ON -D CUDA_SDK_ROOT_DIR=/opt/cuda/samples -D CUDA_TOOLKIT_ROOT_DIR=/opt/cuda/ .options.
I don’t know if that makes a difference.
Don’t get me wrong: it works and that’s great. But, if there’s a way to follow official conventions for uniformity’s sake and for possible more accurate bug reporting, then I’d like to strive to achieve that.
Let me know your thoughts, anyone (especially @brecht).
Can I share a link here? I can post my build on Google Drive for the purposes of telling me if this is ready to share with the community, or not. If it’s cool, then I’ll post immediately.
This options is generally managed by our build profiles, to get a build with the options we use for a release build i recommend using
-C"/blendersourcedirwhereveritmaybe/build_files/cmake/config/blender_release.cmake"
instead
Looking at the CMakeCache.txt that it produces, it is indeed taking care of CUDA… also, I see libGL and libGLU in there, too. Wow. Why didn’t I think to look in this cmake file? Thanks, @LazyDodo !
Let’s see how this build goes.
So, built successfully. Another snag… when I take the build over to a Debian-based machine, when starting ./blender, I get a complaint that is needs GLIBC_2.28. One the latest Linux Mint and Ubuntu, it’s currently GLIBC_2.27.
Also, the build isn’t generating the icons folder or the lib folder, I just realized. So, no libGL.so or libGLU.so, either.
Just to clarify, on the build system and all other systems with Arch [current], everything seems to work fine. Other flavours of Linux have this issue.
Edit: I noticed in CMakeCache.txt that OPENGL_xmesa_INCLUDE_DIR=OPENGL_xmesa_INCLUDE_DIR-NOTFOUND. Is that normal?