How to speed up compiling cycles kernel?

Blender Development Building Blender
#1

Hi,

I’m currently trying to fix some bugs with shader nodes in cycles such as :anchor: T98672 Noise texture shows incorrect behavior for large scale values (blender.org).
Whenever I make the slightest change to code in the cycles kernel it has to recompile the kernel. This is a huge problem for me, as for some reason the cycles kernel takes a tremendous amount of time to compile (about half an hour). Effectively meaning that I can only tweak the code every half an hour.

So I looked at the CPU usage while compiling the kernel vs the rest of the Blender source code and what I found may be a reason as to why the kernel takes so long to compile.

When compiling source code other than the cycles kernel the Visual C++ compiler uses almost 100% of the available CPU processing power.
However when compiling the cycles kernel it’s in the 10-30% CPU processing power range, with most cores simply lying idle.
When compiling the CUDA and OptiX binaries I also ran into the same issues.

How can I make compiling the cycles kernel use all of my CPU processing power or speed up compilation by other means?

#2

For development best to not build things you don’t need, enable WITH_CYCLES_NATIVE_ONLY in cmake to limit the build to only building the kernel you need for you CPU.

similar things can be done for CUDA, by removing the unneeded architectures from the CYCLES_CUDA_BINARIES_ARCH variable.

That being said usually while i’m developing, i develop on just the CPU (with WITH_CYCLES_NATIVE_ONLY=On, WITH_CYCLES_CUDA_BINARIES=Off) and only enable GPU support once the CPU code is in the shape i needs to be in.

2 Likes
#3

Thanks! I tried building with WITH_CYCLES_NATIVE_ONLY=On and WITH_CYCLES_CUDA_BINARIES=Off which shaved down build times to around 10 minutes. Are there any tricks to further shave off build times?

#4

I don’t have a build setup right now, could be we can turn some more things off, what is it spending its time on?

#5

Those files take very long to compile:

kernel.cpp
kernel_avx2.cpp
kernel_avx2.cpp

#6

i’d have to double check, but i don’t think kernel_avx2.cpp is supposed to build when WITH_CYCLES_NATIVE_ONLY is on , just kernel.cpp

#7

@LazyDodo After some experimentation I found out that apparently for WITH_CYCLES_NATIVE_ONLY=ON to work correctly I also have to switch Debug Type to Native Only in Visual Studio.

But despite this it still takes a few minutes to compile a single file called buildinfo.c.

Are there any ways to speed up compilation fot this file?

#8

For that one you can use the nobuildinfo option when calling make.bat for the first time (equivalent to cmake option -DWITH_BUILDINFO=OFF).

When this is disabled you will not get any date/git version info embedded in your build. This is what causes that file to always be re-built and triggers extra work for the compiler:
image

#9

@deadpin Thanks, it worked! Now I can finally focus on writing code instaed of waiting for it to compile.

Are there some other settings that one should generally turn on to speed up compilation?