Building blender with limited number of processors

Hello,

I have an app using make.bat to compile Blender, but I would like to compile it with a limited number of cores just in case I am doing something else on my computer. I just checked the help on make.bat but it doesn’t seem like -j is available like for cmake, is there a way I can change that ? And if possible without recreating the behavior of make.bat and cmake commands :smiley:

Thanks !

make ninja builddir my_build
cd ../my_build
ninja -j5

1 Like

Thanks for the tips.

For some reason I can’t have ninja to work, if I remember correctly that was why I dropped it in the first place. So I have two solutions I guess, either I solve the ninja problem, or maybe there is another way to do the multiprocessing ?

Just in case, the ninja error is just:

ninja: build stopped: subcommand failed.

I read the it could be a memory issue, I have 32Gb of RAM, do you think it is enough (I’m on Windows) ?

Cheers !

I have just 16Gb, but ususally i use lite build and 16 threads.

this has a few problems :

make ninja builddir my_build will still start an initial build using all cores, it be better to skip that initial build all together with make ninja builddir my_build nobuild.

cd ../my_build , perfect the way it is

ninja -j5, at this point the MSVC environment may not have been setup, make.bat placed a helper batch file in the build dir called rebuild.cmd which sets up the environment if not already done, so i’d call rebuild -j5 there.

ninja: build stopped: subcommand failed.

Pretty sure there’s more to it than that, if you provide all output from when you ran ninja until that error we can take a look at what’s going wrong for you.

1 Like

Thanks for the clarity :slight_smile:

Here is the error before the ninja message:

[547/5643] Building CXX object intern\cycles\device\CMakeFiles\cycles_device.dir\oneapi\device_impl.cpp.obj
FAILED: intern/cycles/device/CMakeFiles/cycles_device.dir/oneapi/device_impl.cpp.obj
C:\PROGRA~1\MICROS~2\2022\COMMUN~1\VC\Tools\MSVC\1437~1.328\bin\Hostx64\x64\cl.exe  /nologo /TP -DBLENDER_WIN_APPID=\"blender.4.0\" -DBLENDER_WIN_APPID_FRIENDLY_NAME="\"Blender 4.0\"" -DBOOST_ALL_NO_LIB -DCCL_NAMESPACE_BEGIN="namespace ccl {" -DCCL_NAMESPACE_END=} -DCYCLES_GFLAGS_NAMESPACE=gflags -DCYCLES_RUNTIME_OPTIX_ROOT_DIR=\"\" -DEMBREE_MAJOR_VERSION=4 -DGOOGLE_GLOG_DLL_DECL="" -DNDEBUG -DNOMINMAX -DOIIO_NO_SSE=1 -DOSL_STATIC_BUILD -DOSL_STATIC_LIBRARY -DPATH_GUIDING_LEVEL=5 -DTBB_SUPPRESS_DEPRECATED_MESSAGES=1 -DWIN32 -DWITH_ALEMBIC -DWITH_BLENDER_GUARDEDALLOC -DWITH_CUDA -DWITH_CUDA_DYNLOAD -DWITH_CYCLES_LOGGING -DWITH_EMBREE -DWITH_EMBREE_GPU -DWITH_HIP -DWITH_HIP_DYNLOAD -DWITH_KERNEL_AVX2 -DWITH_KERNEL_SSE2 -DWITH_KERNEL_SSE41 -DWITH_NANOVDB -DWITH_OCIO -DWITH_ONEAPI -DWITH_OPENIMAGEDENOISE -DWITH_OPENSUBDIV -DWITH_OPENVDB -DWITH_OPENVDB_BLOSC -DWITH_OPTIX -DWITH_OSL -DWITH_PATH_GUIDING -DWITH_SYSTEM_PUGIXML -DWITH_TBB_MALLOC -DWITH_USD -D_ALLOW_KEYWORD_MACROS -D_CONSOLE -D_CRT_NONSTDC_NO_DEPRECATE -D_CRT_SECURE_NO_DEPRECATE -D_LIB -D_SCL_SECURE_NO_DEPRECATE -D_USE_MATH_DEFINES -D_WIN32_WINNT=0x603 -D__LITTLE_ENDIAN__ -D__MMX__ -D__SSE2__ -D__SSE__ -IF:\prog\blender_git\blender\intern\cycles\..\..\extern\cuew\include -IF:\prog\blender_git\blender\intern\cycles\..\..\extern\hipew\include -IF:\prog\blender_git\blender\intern\cycles\..\atomic -IF:\prog\blender_git\blender\intern\cycles\device\.. -external:IF:\prog\blender_git\blender\..\lib\win64_vc15\pthreads\include -external:IF:\prog\blender_git\blender\..\lib\win64_vc15\boost\include -external:IF:\prog\blender_git\blender\..\lib\win64_vc15\OpenImageIO\include -external:IF:\prog\blender_git\blender\..\lib\win64_vc15\openexr\include -external:IF:\prog\blender_git\blender\..\lib\win64_vc15\imath\include -external:IF:\prog\blender_git\blender\..\lib\win64_vc15\imath\include\Imath -external:IF:\prog\blender_git\blender\..\lib\win64_vc15\openexr\include\OpenEXR -external:IF:\prog\blender_git\blender\..\lib\win64_vc15\pugixml\include -external:IF:\prog\blender_git\lib\win64_vc15\osl\include -external:I"C:\ProgramData\NVIDIA Corporation\OptiX SDK 7.3.0\include" -external:IF:\prog\blender_git\blender\..\lib\win64_vc15\embree\include -external:IF:\prog\blender_git\blender\..\lib\win64_vc15\OpenImageDenoise\include -external:IF:\prog\blender_git\blender\extern\glog\src\windows -external:IF:\prog\blender_git\blender\extern\gflags\src -external:IF:\prog\blender_git\blender\..\lib\win64_vc15\alembic\include -external:IF:\prog\blender_git\blender\..\lib\win64_vc15\tbb\include -external:IF:\prog\blender_git\blender\..\lib\win64_vc15\openVDB\include -external:IF:\prog\blender_git\blender\..\lib\win64_vc15\openvdb\include -external:IF:\prog\blender_git\blender\..\lib\win64_vc15\opensubdiv\include -external:IF:\prog\blender_git\blender\..\lib\win64_vc15\OpenColorIO\include -external:IF:\prog\blender_git\lib\win64_vc15\openpgl\include -external:IF:\prog\blender_git\lib\win64_vc15\dpcpp\include -external:IF:\prog\blender_git\lib\win64_vc15\dpcpp\include\sycl -external:W0 /W3 /w34062 /w34100 /w34115 /w34189 /w35038 /wd4018 /wd4146 /wd4065 /wd4127 /wd4181 /wd4200 /wd4244 /wd4267 /wd4305 /wd4800 /wd4828 /wd4996 /wd4661 /wd4848 /we4013 /we4133 /we4431 /we4033 /DWIN32 /D_WINDOWS /W3  /EHsc /nologo /J /Gd /MP /EHsc /bigobj /Zc:inline /permissive- -openmp /Zc:__cplusplus    /fp:fast -D_CRT_SECURE_NO_WARNINGS /GS- /jumptablerdata  /O2 /Ob2 /DNDEBUG /MD /Zi /Ox -std:c++17 /utf-8 /experimental:external /external:I F:/prog/blender_git/blender/../lib/win64_vc15 /external:W0 /showIncludes /Fointern\cycles\device\CMakeFiles\cycles_device.dir\oneapi\device_impl.cpp.obj /Fdintern\cycles\device\CMakeFiles\cycles_device.dir\cycles_device.pdb /FS -c F:\prog\blender_git\blender\intern\cycles\device\oneapi\device_impl.cpp
F:\prog\blender_git\lib\win64_vc15\dpcpp\include\sycl/detail/property_list_base.hpp(123): error C2039: 'iter_swap': is not a member of 'std'
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.37.32822\include\bitset(22): note: see declaration of 'std'
F:\prog\blender_git\lib\win64_vc15\dpcpp\include\sycl/detail/property_list_base.hpp(123): error C3861: 'iter_swap': identifier not found
Alignment of class vec is not in accordance with SYCL specification requirements, a limitation of the MSVC compiler(Error C2719).Requested alignment applied, limited at 64.
[548/5643] Building CXX object intern\cycles\integrator\CMakeFiles\cycles_integrator.dir\tile.cpp.obj
[549/5643] Building CXX object intern\cycles\integrator\CMakeFiles\cycles_integrator.dir\denoiser_gpu.cpp.obj
[550/5643] Building CXX object intern\cycles\integrator\CMakeFiles\cycles_integrator.dir\denoiser_oidn.cpp.obj
[551/5643] Building CXX object intern\cycles\integrator\CMakeFiles\cycles_integrator.dir\denoiser_optix.cpp.obj
[552/5643] Building CXX object intern\cycles\device\CMakeFiles\cycles_device.dir\oneapi\device.cpp.obj
Alignment of class vec is not in accordance with SYCL specification requirements, a limitation of the MSVC compiler(Error C2719).Requested alignment applied, limited at 64.
[553/5643] Building CXX object intern\cycles\device\CMakeFiles\cycles_device.dir\oneapi\queue.cpp.obj
Alignment of class vec is not in accordance with SYCL specification requirements, a limitation of the MSVC compiler(Error C2719).Requested alignment applied, limited at 64.
[554/5643] Building CXX object intern\cycles\integrator\CMakeFiles\cycles_integrator.dir\pass_accessor.cpp.obj
[555/5643] Building CXX object intern\cycles\integrator\CMakeFiles\cycles_integrator.dir\path_trace.cpp.obj
[556/5643] Building CXX object intern\cycles\integrator\CMakeFiles\cycles_integrator.dir\pass_accessor_cpu.cpp.obj
ninja: build stopped: subcommand failed.
Error during build

For the rebuild method, if I understand correctly, I have to build Blender at least one time with make.bat the first time, and then I can setup the cores with rebuild -j5, and then run make.bat again ?

can you try this patch?

diff --git a/intern/cycles/device/oneapi/device_impl.cpp b/intern/cycles/device/oneapi/device_impl.cpp
index 4928b1124fc..f3bcc1248ee 100644
--- a/intern/cycles/device/oneapi/device_impl.cpp
+++ b/intern/cycles/device/oneapi/device_impl.cpp
@@ -3,7 +3,7 @@
  * SPDX-License-Identifier: Apache-2.0 */

 #ifdef WITH_ONEAPI
-
+#  include <algorithm>
 #  include "device/oneapi/device_impl.h"

 #  include "util/debug.h"
1 Like

Thanks a lot, ninja made it through :slight_smile:
I just have one message at the end:

[8/5076] Building CXX object intern\cycles\device\CMakeFiles\cycles_device.dir\oneapi\device_impl.cpp.obj
Alignment of class vec is not in accordance with SYCL specification requirements, a limitation of the MSVC compiler(Error C2719).Requested alignment applied, limited at 64.
[5075/5076] Linking CXX executable bin\blender.exe

I’ll have a try with rebuild.cmd and let you know if I can make it work.

Is that a diff I would always have to apply before building ? Or was it something planned to be merged anyway ?