the cuda compiler tends not to ship with most distribution, you need to grab the cuda toolkit from nvidia.
CUDA 10 can be found in Arch’s package repos. It installed under /opt just fine. I was finally able to compile with everything I need (except OSL, which I currently don’t need) with the following make command. You could easily add these into CMakeCache.txt, for those who choose to.
make -j12 BUILD_CMAKE_ARGS="-U *SNDFILE* -U *PYTHON* -U *BOOST* -U *Boost* -U *OPENCOLORIO* -U *OPENEXR* -U *OPENIMAGEIO* -U *LLVM* -U *CYCLES* -U *OPENSUBDIV* -U *OPENVDB* -U *COLLADA* -U *FFMPEG* -U *ALEMBIC* -D WITH_CODEC_SNDFILE=ON -D PYTHON_VERSION=3.7 -D WITH_OPENCOLORIO=ON -D WITH_LLVM=ON -D LLVM_VERSION=6.0.1 -D LLVM_ROOT_DIR=/opt/lib/llvm -D LLVM_STATIC=ON -D WITH_OPENSUBDIV=ON -D WITH_OPENVDB=ON -D WITH_OPENVDB_BLOSC=ON -D WITH_ALEMBIC=ON -D ALEMBIC_ROOT_DIR=/opt/lib/alembic -D WITH_CODEC_FFMPEG=ON -D FFMPEG_LIBRARIES='avformat;avcodec;avutil;avdevice;swscale;swresample;lzma;rt;theora;theoradec;theoraenc;vorbis;vorbisenc;vorbisfile;ogg;x264;openjp2' -D WITH_OPENVDB_3_ABI_COMPATIBLE=ON -D WITH_CYCLES_CUDA_BINARIES=ON -D CUDA_SDK_ROOT_DIR=/opt/cuda/samples -D CUDA_TOOLKIT_ROOT_DIR=/opt/cuda/" full
Note that the WITH_OPENVDB_3_ABI_COMPATIBLE=ON is specifically for building on Arch because OpenVDB (I think) ships with V3 compatibility built in, apparently.
Good to hear you got it going, bonus points for posting how so anyone in the future can benefit from it!
Two things: those settings can be passed via initial cmake invocation, and that -j12 is not really working.
To generate the makefiles with some options:
cd /dir/for/build
cmake /path/to/source -DWITH_WHATEVER=onIf you run make in the build tree, -jN will work, because you are using the Makefiles generated by cmake. If you run make in the source (ie via GNUMakefile, as make full, etc), it is ignored (say you want -j2 but have 16 cores… you get 16, not 2).
Yes, you are correct. I will note that, when I eventually write a HowTo. Thanks for pointing that out.
While I’m at it, let me ask… Since I’m learning how to make a completely static and portable build of this branch (or any branch, for that matter), I’m looking at blender/build_files/buildbot/config/blender_linux.cmake for some idea on what flags I need and libraries should be included. Am I going in the right direction? Any sage advice on this endeavour?
For a completely static build, run make deps and then make. The static libraries are automatically picked up by the build system. This is the mechanism that is used for building static libraries on the buildbot and in releases.
It has not been tested on as many distributions as install_deps.sh and may need some manual tweaking.
1 Like
Thanks, Brecht. Do I run it under …/blender? I can’t run it under the branch build folder, as it doesn’t recognize the make deps command, which makes sense. If I run it under …/blender, can I still build a static build under the branch build folder?
Yes, you run it from ../blender. It will use the ../build_linux/deps directory and not conflict with any other builds.
Ran it from ../blender. Stopped on errors at 23%.
[23%] Completed 'external_ssl'
[ 23%] Built target external_ssl
make[1]: *** [Makefile:130: all] Error 2
make: *** [GNUmakefile:306: deps] Error 2
Did it die on external_ssl or sqlite? I looked at ../build_linux/deps/CMakeLists.txt to see the order of the builds, but I don’t know where it’s choking.
How do I diagnose and correct? I’m running this on the most current Arch with kernel 5.0.4.
Easiest is to just run make deps a few times until only the libraries that fail to build remain, then scroll back in the log. If you manually run make from ../build_linux/deps/ you can do a single threaded build too.
So, I ran it a few times. Then ran it from build-linux/deps on a single thread to single it out further and it appears to be choking on external_sqlite. Here are the last few throws:
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/alter.c:794:31: error: ‘FuncDef’ {aka ‘struct FuncDef’} has no member named ‘pSchema’
pNew->pSchema = db->aDb[iDb].pSchema;
^
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/alter.c:794:17: warning: assignment to ‘Schema *’ {aka ‘struct Schema *’} from incompatible pointer type ‘FuncDef *’ {aka ‘struct FuncDef *’} [-Wincompatible-pointer-types]
pNew->pSchema = db->aDb[iDb].pSchema;
^
make[3]: *** [Makefile:743: alter.lo] Error 1
make[2]: *** [CMakeFiles/external_sqlite.dir/build.make:115: build/sqlite/src/external_sqlite-stamp/external_sqlite-build] Error 2
make[1]: *** [CMakeFiles/Makefile2:1360: CMakeFiles/external_sqlite.dir/all] Error 2
make: *** [Makefile:130: all] Error 2
Further up the messages, there are errors, such as, error: unknown type name ‘sqlite_int64’. Don’t know what’s a showstopper and what’s not.
Thoughts?
The important errors to find the issue are the first ones, not the last ones.
Here is the first batch of errors.
In file included from /home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/alter.c:15:
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/sqliteInt.h:734:9: error: unknown type name ‘sqlite_int64’
typedef sqlite_int64 i64; /* 8-byte signed integer */
^~~~~~~~~~~~
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/sqliteInt.h:735:9: error: unknown type name ‘sqlite_uint64’
typedef sqlite_uint64 u64; /* 8-byte unsigned integer */
^~~~~~~~~~~~~
In file included from /home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/sqliteInt.h:1125,
from /home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/alter.c:15:
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/btree.h:46:3: error: unknown type name ‘sqlite3_vfs’
sqlite3_vfs *pVfs, /* VFS to use with this b-tree */
^~~~~~~~~~~
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/btree.h:48:3: error: unknown type name ‘sqlite3’
sqlite3 *db, /* Associated database connection */
^~~~~~~
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/btree.h:69:39: error: unknown type name ‘sqlite3_int64’
int sqlite3BtreeSetMmapLimit(Btree*,sqlite3_int64);
^~~~~~~~~~~~~
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/btree.h:293:3: error: unknown type name ‘sqlite3_int64’
sqlite3_int64 nKey; /* Size of pKey for indexes. PRIMARY KEY for tabs */
^~~~~~~~~~~~~
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/btree.h:295:3: error: unknown type name ‘sqlite3_value’
sqlite3_value *aMem; /* First of nMem value in the unpacked pKey */
^~~~~~~~~~~~~
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/btree.h:356:29: error: unknown type name ‘sqlite3’
void sqlite3BtreeEnterAll(sqlite3*);
^~~~~~~
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/btree.h:371:29: error: unknown type name ‘sqlite3’
void sqlite3BtreeLeaveAll(sqlite3*);
^~~~~~~
In file included from /home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/sqliteInt.h:1126,
from /home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/alter.c:15:
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/vdbe.h:55:5: error: unknown type name ‘sqlite3_context’
sqlite3_context *pCtx; /* Used when p4type is P4_FUNCCTX */
^~~~~~~~~~~~~~~
In file included from /home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/sqliteInt.h:1126,
from /home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/alter.c:15:
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/vdbe.h:236:29: error: unknown type name ‘sqlite3’
void sqlite3VdbeClearObject(sqlite3*,Vdbe*);
^~~~~~~
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/vdbe.h:253:1: error: unknown type name ‘sqlite3’
sqlite3 *sqlite3VdbeDb(Vdbe*);
^~~~~~~
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/vdbe.h:258:1: error: unknown type name ‘sqlite3_value’; use ‘struct’ keyword to refer to the type
sqlite3_value *sqlite3VdbeGetBoundValue(Vdbe*, int, u8);By the time it gets to external_sqlite, 78% of the libs are built. For some of them, the solution was simply to install nasm (yasm didn’t take over and I don’t know if I have to stomach to find flags on that many libs that required flags).
You can try removing the ../build_linux/deps/build/sqlite folder and building that library again from the start. It’s hard to tell anything from that snippet, would need to see the complete log, the actual problem is probably even earlier.
Hi brecht.
If “blender” folder is inside other folder whose name contains spaces, then “make deps” does not work. For example if the path is “/Disk/Blender Git/blender” and I run “make deps” inside “blender” folder, it gives an error about “/Disk/Blender Git/blender/Git/lib/linux_x86_64” does not exist. If I rename and remove the space, for example “/Disk/Blender_Git/blender”, it works. Could this be corrected to support folder names with spaces?
I removed the sqlite folder, as you suggested, and ran make again. These are the errors:
[ 78%] Performing download step (download, verify and extract) for 'external_sqlite'
-- Downloading...
dst='/home/haig/blender-git/build_linux/deps/build/sqlite/src/sqlite-src-3240000.zip'
timeout='none'
-- Using src='https://www.sqlite.org/2018/sqlite-src-3240000.zip'
-- verifying file...
file='/home/haig/blender-git/build_linux/deps/build/sqlite/src/sqlite-src-3240000.zip'
-- Downloading... done
-- extracting...
src='/home/haig/blender-git/build_linux/deps/build/sqlite/src/sqlite-src-3240000.zip'
dst='/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite'
-- extracting... [tar xfz]
-- extracting... [analysis]
-- extracting... [rename]
-- extracting... [clean up]
-- extracting... done
[ 78%] No patch step for 'external_sqlite'
[ 78%] No update step for 'external_sqlite'
[ 78%] Performing configure step for 'external_sqlite'
.
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for ar... ar
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
checking for a BSD-compatible install... /usr/bin/install -c
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... no
checking for int8_t... yes
checking for int16_t... yes
checking for int32_t... yes
checking for int64_t... yes
checking for intptr_t... yes
checking for uint8_t... yes
checking for uint16_t... yes
checking for uint32_t... yes
checking for uint64_t... yes
checking for uintptr_t... yes
checking for sys/types.h... (cached) yes
checking for stdlib.h... (cached) yes
checking for stdint.h... (cached) yes
checking for inttypes.h... (cached) yes
checking malloc.h usability... yes
checking malloc.h presence... yes
checking for malloc.h... yes
checking for fdatasync... yes
checking for gmtime_r... yes
checking for isnan... yes
checking for localtime_r... yes
checking for localtime_s... no
checking for malloc_usable_size... yes
checking for strchrnul... yes
checking for usleep... yes
checking for utime... yes
checking for pread... yes
checking for pread64... yes
checking for pwrite... yes
checking for pwrite64... yes
checking for tclsh8.7... no
checking for tclsh8.6... no
checking for tclsh8.5... no
checking for tclsh... no
Warning: can't find tclsh - defaulting to non-amalgamation build.
./configure: line 7595: tclsh: command not found
configure: Version set to 3.24
configure: Release set to 3.24.0
configure: Version number set to 3024000
checking whether to support threadsafe operation... yes
checking for library containing pthread_create... -lpthread
checking for library containing pthread_mutexattr_init... none required
checking whether to support shared library linked as release mode or not... no
checking whether to use an in-ram database for temporary tables... no
checking if executables have the .exe suffix... unknown
checking for Tcl configuration... ./configure: line 7975: tclsh: command not found
./configure: line 7989: tclsh: command not found
configure: WARNING: Can't find Tcl configuration definitions
configure: WARNING: *** Without Tcl the regression tests cannot be executed ***
configure: WARNING: *** Consider using --with-tcl=... to define location of Tcl ***
checking for library containing readline... -ledit
checking for library containing fdatasync... none required
checking zlib.h usability... yes
checking zlib.h presence... yes
checking for zlib.h... yes
checking for library containing deflate... -lz
checking for library containing dlopen... -ldl
checking whether to support MEMSYS5... no
checking whether to support MEMSYS3... no
checking for library containing log... -lm
checking for library containing log... (cached) -lm
configure: creating ./config.status
config.status: creating Makefile
config.status: creating sqlite3.pc
config.status: creating config.h
config.status: executing libtool commands
[ 79%] Performing build step for 'external_sqlite'
/bin/sh: tclsh: command not found
make[3]: *** [Makefile:1003: sqlite3.h] Error 127
make[2]: *** [CMakeFiles/external_sqlite.dir/build.make:115: build/sqlite/src/external_sqlite-stamp/external_sqlite-build] Error 2
make[1]: *** [CMakeFiles/Makefile2:1360: CMakeFiles/external_sqlite.dir/all] Error 2
make: *** [Makefile:130: all] Error 2There’s that warning about tcl, but I don’t really know what that means.
So, I installed tcl, removed sqlite folder, and ran make again. Now, I get:
Libraries have been installed in:
/home/haig/blender-git/build_linux/deps/Release/sqlite/lib
If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
- add LIBDIR to the `LD_LIBRARY_PATH' environment variable
during execution
- add LIBDIR to the `LD_RUN_PATH' environment variable
during linking
- use the `-Wl,-rpath -Wl,LIBDIR' linker flag
- have your system administrator add LIBDIR to `/etc/ld.so.conf'
See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
/usr/bin/install: cannot change permissions of ‘/usr/lib/tcl8.6/sqlite3’: No such file or directory
make[3]: *** [Makefile:1350: tcl_install] Error 1
make[2]: *** [CMakeFiles/external_sqlite.dir/build.make:74: build/sqlite/src/external_sqlite-stamp/external_sqlite-install] Error 2
make[1]: *** [CMakeFiles/Makefile2:1360: CMakeFiles/external_sqlite.dir/all] Error 2
make: *** [Makefile:130: all] Error 2Edit: usr/lib/tcl8.6/sqlite3 doesn’t exist. Hm… does a lib need to be linked? Env var set to something, maybe?
You can run the NPROCS=1 make for the GNUMakefile. It would be better if it understood -j and passed it, instead of reinventing the wheel, of course.
Actually, running make from inside ../build_linux/deps takes the -j1 option, I discovered with all your help.
I think you need to apt install tcl. Never heard of this being needed to build our libraries, but appears sqlite needs it.
Yes. I did install it, as I mentioned, and get the second set of errors (first set are gone after tcl) that I posted because /usr/lib/tcl8.6/sqlite3 isn’t present. I searched the system and couldn’t find sqlite3 anywhere, except in a python folder (that can’t be it).
So, what I did was (and this is on Arch, mind you) sudo mkdir /usr/lib/tcl8.6/sqlite3 and changed its ownership to user. Ran make again and voila! sqlite is built.