Blender 2.8: Cycles OptiX packaging

Dear Developers,

I’m involved in packaging Blender for the openSUSE eco system, based here and was asked to add OptiX support for Cycles. Further investigation revealed, that - unlike CUDA - OptiX support requires a working CUDA stack during build time, which in turn requires a working nvidia driver installed.

A public packaging/build infrastructure like OBS, which generates all builds of the distribution cannot provide such an environment for a multitude of reasons (starting with non-free NVidia driver code).

The best way to deal with this issue would be a deferred compilation of Cycles IMHO.

Is there anything, I’m missing? Opinions?

AFAIK to build Blender with Optix enabled is need use CUDA SDK and Optix Library, i build my Blender on VM without OpenGL support and surely without NVidia drivers installed, yoyu can check my compiled Blender from here. Fork is not mine, i only compiled this Blender branch/fork.

Building Blender with Optix support does not require a working CUDA stack or NVIDIA driver any more than CUDA support.

What it requires is the Optix SDK, and specifically the header files in the SDK.
https://wiki.blender.org/wiki/Building_Blender/CUDA

Unfortunately downloading that SDK requires singing up for an NVIDIA developer account, and redistributing the headers is not permitted by the license.

1 Like

Hey, thanks for the hints, @NiCapp and @brecht, much appreciated.

My build mods:

Index: blender-git.spec
===================================================================
--- blender-git.spec    (revision 111)
+++ blender-git.spec    (working copy)
@@ -58,6 +58,7 @@
 Source0:        %{_origname}-%{version}.tar.xz
 Source1:        %{name}.appdata.xml
 Source2:        SUSE-NVIDIA-GPU-rendering.txt
+Source3:        nvidia-optix-7.0.tar.xz
 %if %{with python_36}
 Patch1:         make_python_3.6_compatible.patch
 %endif
@@ -216,7 +217,7 @@
 %lang_package
 
 %prep
-%setup -q -n %{_origname}-%{version}
+%setup -q -n %{_origname}-%{version} -a3
 %if %{with python_36}
 %patch1 -p1
 %endif
@@ -357,7 +358,9 @@
       -DCYCLES_CUDA_BINARIES:BOOL=ON \
       -DCYCLES_CUBIN_COMPILER:BOOL=OFF \
       -DCYCLES_CUDA_BINARIES_ARCH="sm_30;sm_35;sm_37;sm_50;sm_52;sm_60;sm_61;sm_70;sm_75" \
-      -DWITH_CYCLES_DEVICE_OPTIX:BOOL=OFF
+      -DWITH_CYCLES_DEVICE_OPTIX:BOOL=ON \
+      -DOPTIX_ROOT_DIR:STRING="$(pwd)/nvidia-optix-7.0" \
+      -DOPTIX_INCLUDE_DIR:STRING="$(pwd)/nvidia-optix-7.0/include"
 
 make %{?_smp_mflags}

that result in:

[   14s] -- Found OptiX: /home/abuild/rpmbuild/BUILD/blender-2.83~git.20200403T103721.fe7ea8a24c8/build/nvidia-optix-7.0/include  

But fails with:

[   22s] [  2%] Generating kernel_optix.ptx
[   22s] cd "/home/abuild/rpmbuild/BUILD/blender-2.83~git.20200403T103721.fe7ea8a24c8/intern/cycles/kernel" && --ptx -arch=sm_30 -I /home/abuild/rpmbuild/BUILD/blender-2.83~git.20200403T103721.fe7ea8a24c8/build/nvidia-optix-7.0/include -I /home/abuild/rpmbuild/BUILD/blender-2.83~git.20200403T103721.fe7ea8a24c8/intern/cycles/kernel/.. -I /home/abuild/rpmbuild/BUILD/blender-2.83~git.20200403T103721.fe7ea8a24c8/intern/cycles/kernel/kernels/cuda --use_fast_math -o /home/abuild/rpmbuild/BUILD/blender-2.83~git.20200403T103721.fe7ea8a24c8/build/intern/cycles/kernel/kernel_optix.ptx kernels/optix/kernel_optix.cu
[   22s] /bin/sh: --ptx: command not found
[   22s] make[2]: *** [intern/cycles/kernel/CMakeFiles/cycles_kernel_optix.dir/build.make:284: intern/cycles/kernel/kernel_optix.ptx] Error 127
[   22s] make[2]: Leaving directory '/home/abuild/rpmbuild/BUILD/blender-2.83~git.20200403T103721.fe7ea8a24c8/build'
[   22s] make[1]: *** [CMakeFiles/Makefile2:3621: intern/cycles/kernel/CMakeFiles/cycles_kernel_optix.dir/all] Error 2

The build tries to call nvcc (I guess), but that isn’t available at build time. This is the reason for my statement above.

@brecht Without defining OPTIX_INCLUDE_DIR, the result is:

[   14s] -- Could NOT find OptiX (missing: OPTIX_INCLUDE_DIR) 
[   14s] -- OptiX not found, disabling it from Cycles

While at it, due to the license issues, my first attempt was to supply the SDK include tree, which might be a feasible workaround, but that fails in the same way…

Ah yes, we don’t yet support runtime compilation for Optix.

I plan to add that still, but not exactly sure when.

These seems cmake messages though, not runtime builds, looks like the NVCC is not found, try setting -DCUDA_NVCC_EXECUTABLE=/where/ever/nvcc/lives/at/build/time/nvcc

That would be great, @brecht.

Guess, basic infrastructure is in place but as always, the devil is in the details…
Will keep the bug on hold until then. A brief notice, when you’re ready for testing, is much appreciated.

Thanks, @LazyDodo, the matter is, at the time and place, when and where such distribution packages are build, nvcc cannot exist for a multitude of reasons (with the foremost: legal aka NVidia License Agreement). In fact, that’s the way, CUDA works for Blender: whenever you attempt to run Cycles on a new Blender build, it attempts to compile the module first, which is - as usual being expected from this project :wink: - a very smart way to solve such issues. Of course, this comes with a prize: these modules must be created even more carefully than others, given the variants (CUDA versions) available today.

Could the nvidia driver exist? cause we do have a compiler based on the nvrtc library

Soory, @LazyDodo, I don’t get your question. As written before, it’s not possible to officially build Blender with the drivers installed for any Linux distribution. That is a restriction of the build systems, that disallow builds with non-free drivers installed (and typically are done in virtualized build environments, where such drivers won’t fit). Specialized builds are possible of course, but these would make the whole package non-free, since NVidia’s License Agreement forbids publishing the sources (headers et al).

Take it as an unfortunate legal and technical interaction.

At the moment, @brecht has mangled the OptiX integration into Cycles into a runtime dependency, all will be well again. You will not notice anything apart from a small delay on the first usage. And for the impatient ones, you can always use the precompiled packages from blender.org.

I took a quick look, D7400 does the trick on windows, but i don’t have linux to test on.

edit: brecht was quick on the review, diff has landed in master already.