Help with compiling sculpt-mode-features branch

I’m very interested in testing Pablo Dobarro’s new sculpt features on Linux systems, which means I have to build it from scratch. I build Blender from git, every day, without issues. However, I’m running into trouble while building from the sculpt-mode-features branch. Before diving into the exact error that occurs anywhere between 25%-27% of the way down (with no errors on deps), which is ambiguous, could someone please instruct me on the proper way to checkout the branch, update the repo, and build?

Right now, I just switch to git checkout sculpt-mode-features and then run a make update before running make -j12 BUILD_CMAKE_ARGS="-D WITH_OPENVDB_3_ABI_COMPATIBLE=ON" full. The extra OpenVDB flag is necessary to compile Blender 2.80 on Arch.

To be honest, I’ve never built from any branch other than origin for Blender, so I’m very likely doing something wrong. Any help would be appreciated.

1 Like

Branches should work as others. Here it built as master does. Maybe the errors are trying to tell you something.

You could make sure WITH_OPENVDB_3_ABI_COMPATIBLE=ON appears in the CMakeCache.txt, and then you don’t need to pass it.
Also try -j1 or even -j100 and see what it does (because based in how everything is setup, I doubt it does what you expect).

For developing, debug is more useful than full (more info in crashes, eg).

@gsrb3d Thanks for the advice. I did start clean and turned on WITH_OPENVDB_3_ABI_COMPATIBLE in CMakeCache.txt. It built to 100% and threw an error:

now run: "make install" to copy runtime files and scripts to 2.80 [100%] Built target blender executing: /home/haig/blender-git/sculpt-mode-features/bin/blender --help /home/haig/blender-git/sculpt-mode-features/bin/blender: error while loading shared libraries: liboslcomp.so.1.9: cannot open shared object file: No such file or directory Traceback (most recent call last): File "/home/haig/blender-git/blender/doc/manpage/blender.1.py", line 55, in <module> blender_help = subprocess.check_output(cmd).decode(encoding="utf-8") File "/usr/lib/python3.7/subprocess.py", line 395, in check_output **kwargs).stdout File "/usr/lib/python3.7/subprocess.py", line 487, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command '['/home/haig/blender-git/sculpt-mode-features/bin/blender', '--help']' returned non-zero exit status 127. make[2]: *** [source/creator/CMakeFiles/blender_man_page.dir/build.make:57: source/creator/CMakeFiles/blender_man_page] Error 1 make[1]: *** [CMakeFiles/Makefile2:7652: source/creator/CMakeFiles/blender_man_page.dir/all] Error 2 make: *** [Makefile:163: all] Error 2

Looking at that, one would think that I didn’t give a path to OSL_ROOT_DIR=/opt/lib/osl, but looking at the CMakeCache.txt, I see that I have.

Ideas?

idea: do you need osl? if not just turn it off in cmake and sidestep the issue.

Successfully built. Turned off osl. Can use the experimental sculpt features.

However, my CUDA devices are not detected. I can live without OSL, but not without CUDA. Any idea how I can diagnose what’s missing?

make sure WITH_CYCLES_CUDA_BINARIES is on in cmake?

Turned on WITH_CYCLES_CUDA_BINARIES. Now, I get CUDA_SDK_ROOT_DIR-NOTFOUND and CUDA_TOOLKIT_ROOT_DIR-NOTFOUND in CMakeCache.txt. I don’t know which directories to find those in on my system. Any way I can find out?

Blender builds, once again, but without seeing any CUDA devices.

the cuda compiler tends not to ship with most distribution, you need to grab the cuda toolkit from nvidia.

CUDA 10 can be found in Arch’s package repos. It installed under /opt just fine. I was finally able to compile with everything I need (except OSL, which I currently don’t need) with the following make command. You could easily add these into CMakeCache.txt, for those who choose to.

make -j12 BUILD_CMAKE_ARGS="-U *SNDFILE* -U *PYTHON* -U *BOOST* -U *Boost* -U *OPENCOLORIO* -U *OPENEXR* -U *OPENIMAGEIO* -U *LLVM* -U *CYCLES* -U *OPENSUBDIV* -U *OPENVDB* -U *COLLADA* -U *FFMPEG* -U *ALEMBIC* -D WITH_CODEC_SNDFILE=ON -D PYTHON_VERSION=3.7 -D WITH_OPENCOLORIO=ON -D WITH_LLVM=ON -D LLVM_VERSION=6.0.1 -D LLVM_ROOT_DIR=/opt/lib/llvm -D LLVM_STATIC=ON -D WITH_OPENSUBDIV=ON -D WITH_OPENVDB=ON -D WITH_OPENVDB_BLOSC=ON -D WITH_ALEMBIC=ON -D ALEMBIC_ROOT_DIR=/opt/lib/alembic -D WITH_CODEC_FFMPEG=ON -D FFMPEG_LIBRARIES='avformat;avcodec;avutil;avdevice;swscale;swresample;lzma;rt;theora;theoradec;theoraenc;vorbis;vorbisenc;vorbisfile;ogg;x264;openjp2' -D WITH_OPENVDB_3_ABI_COMPATIBLE=ON -D WITH_CYCLES_CUDA_BINARIES=ON -D CUDA_SDK_ROOT_DIR=/opt/cuda/samples -D CUDA_TOOLKIT_ROOT_DIR=/opt/cuda/" full

Note that the WITH_OPENVDB_3_ABI_COMPATIBLE=ON is specifically for building on Arch because OpenVDB (I think) ships with V3 compatibility built in, apparently.

Good to hear you got it going, bonus points for posting how so anyone in the future can benefit from it!

Two things: those settings can be passed via initial cmake invocation, and that -j12 is not really working.

To generate the makefiles with some options:

cd /dir/for/build
cmake /path/to/source -DWITH_WHATEVER=on

If you run make in the build tree, -jN will work, because you are using the Makefiles generated by cmake. If you run make in the source (ie via GNUMakefile, as make full, etc), it is ignored (say you want -j2 but have 16 cores… you get 16, not 2).

Yes, you are correct. I will note that, when I eventually write a HowTo. Thanks for pointing that out.

While I’m at it, let me ask… Since I’m learning how to make a completely static and portable build of this branch (or any branch, for that matter), I’m looking at blender/build_files/buildbot/config/blender_linux.cmake for some idea on what flags I need and libraries should be included. Am I going in the right direction? Any sage advice on this endeavour?

For a completely static build, run make deps and then make. The static libraries are automatically picked up by the build system. This is the mechanism that is used for building static libraries on the buildbot and in releases.

It has not been tested on as many distributions as install_deps.sh and may need some manual tweaking.

1 Like

Thanks, Brecht. Do I run it under …/blender? I can’t run it under the branch build folder, as it doesn’t recognize the make deps command, which makes sense. If I run it under …/blender, can I still build a static build under the branch build folder?

Yes, you run it from ../blender. It will use the ../build_linux/deps directory and not conflict with any other builds.

Ran it from ../blender. Stopped on errors at 23%.

[23%] Completed 'external_ssl'
[ 23%] Built target external_ssl
make[1]: *** [Makefile:130: all] Error 2
make: *** [GNUmakefile:306: deps] Error 2

Did it die on external_ssl or sqlite? I looked at ../build_linux/deps/CMakeLists.txt to see the order of the builds, but I don’t know where it’s choking.

How do I diagnose and correct? I’m running this on the most current Arch with kernel 5.0.4.

Easiest is to just run make deps a few times until only the libraries that fail to build remain, then scroll back in the log. If you manually run make from ../build_linux/deps/ you can do a single threaded build too.

So, I ran it a few times. Then ran it from build-linux/deps on a single thread to single it out further and it appears to be choking on external_sqlite. Here are the last few throws:

/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/alter.c:794:31: error: ‘FuncDef’ {aka ‘struct FuncDef’} has no member named ‘pSchema’
pNew->pSchema = db->aDb[iDb].pSchema;
^
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/alter.c:794:17: warning: assignment to ‘Schema *’ {aka ‘struct Schema *’} from incompatible pointer type ‘FuncDef *’ {aka ‘struct FuncDef *’} [-Wincompatible-pointer-types]
pNew->pSchema = db->aDb[iDb].pSchema;
^
make[3]: *** [Makefile:743: alter.lo] Error 1
make[2]: *** [CMakeFiles/external_sqlite.dir/build.make:115: build/sqlite/src/external_sqlite-stamp/external_sqlite-build] Error 2
make[1]: *** [CMakeFiles/Makefile2:1360: CMakeFiles/external_sqlite.dir/all] Error 2
make: *** [Makefile:130: all] Error 2

Further up the messages, there are errors, such as, error: unknown type name ‘sqlite_int64’. Don’t know what’s a showstopper and what’s not.

Thoughts?

The important errors to find the issue are the first ones, not the last ones.

Here is the first batch of errors.

In file included from /home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/alter.c:15:
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/sqliteInt.h:734:9: error: unknown type name ‘sqlite_int64’
 typedef sqlite_int64 i64;          /* 8-byte signed integer */
         ^~~~~~~~~~~~
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/sqliteInt.h:735:9: error: unknown type name ‘sqlite_uint64’
 typedef sqlite_uint64 u64;         /* 8-byte unsigned integer */
         ^~~~~~~~~~~~~
In file included from /home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/sqliteInt.h:1125,
                 from /home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/alter.c:15:
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/btree.h:46:3: error: unknown type name ‘sqlite3_vfs’
   sqlite3_vfs *pVfs,       /* VFS to use with this b-tree */
   ^~~~~~~~~~~
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/btree.h:48:3: error: unknown type name ‘sqlite3’
   sqlite3 *db,             /* Associated database connection */
   ^~~~~~~
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/btree.h:69:39: error: unknown type name ‘sqlite3_int64’
   int sqlite3BtreeSetMmapLimit(Btree*,sqlite3_int64);
                                       ^~~~~~~~~~~~~
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/btree.h:293:3: error: unknown type name ‘sqlite3_int64’
   sqlite3_int64 nKey;     /* Size of pKey for indexes.  PRIMARY KEY for tabs */
   ^~~~~~~~~~~~~
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/btree.h:295:3: error: unknown type name ‘sqlite3_value’
   sqlite3_value *aMem;    /* First of nMem value in the unpacked pKey */
   ^~~~~~~~~~~~~
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/btree.h:356:29: error: unknown type name ‘sqlite3’
   void sqlite3BtreeEnterAll(sqlite3*);
                             ^~~~~~~
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/btree.h:371:29: error: unknown type name ‘sqlite3’
   void sqlite3BtreeLeaveAll(sqlite3*);
                             ^~~~~~~
In file included from /home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/sqliteInt.h:1126,
                 from /home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/alter.c:15:
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/vdbe.h:55:5: error: unknown type name ‘sqlite3_context’
     sqlite3_context *pCtx; /* Used when p4type is P4_FUNCCTX */
     ^~~~~~~~~~~~~~~
In file included from /home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/sqliteInt.h:1126,
                 from /home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/alter.c:15:
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/vdbe.h:236:29: error: unknown type name ‘sqlite3’
 void sqlite3VdbeClearObject(sqlite3*,Vdbe*);
                             ^~~~~~~
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/vdbe.h:253:1: error: unknown type name ‘sqlite3’
 sqlite3 *sqlite3VdbeDb(Vdbe*);
 ^~~~~~~
/home/haig/blender-git/build_linux/deps/build/sqlite/src/external_sqlite/src/vdbe.h:258:1: error: unknown type name ‘sqlite3_value’; use ‘struct’ keyword to refer to the type
 sqlite3_value *sqlite3VdbeGetBoundValue(Vdbe*, int, u8);

By the time it gets to external_sqlite, 78% of the libs are built. For some of them, the solution was simply to install nasm (yasm didn’t take over and I don’t know if I have to stomach to find flags on that many libs that required flags).