How to solve problems related to gcc and CUDA?

My system is Kubuntu 20.04 (pre beta release) updated from Kubuntu 18.04.
I have gcc versions 7 and 9 installed (v10 was also originally installed but I uninstalled it).
If I compile Blender from master with CUDA binaries, it ends with an error related to “-- unsupported GNU version! gcc versions later than 8 are not supported!”. If I open CMakeCache.txt with cmake-gui, and I search for gcc, apparently it is trying to use gcc version 9. I replace 9 with 7 in all the related gcc, Configure, Generate and build, but the error of “-- unsupported GNU version! gcc versions later than 8 are not supported!” It is still happening.

Now, building without CUDA binaries for it to compile CUDA kernel dynamically… Opening blender and when rendering with GPU I get similar error related to gcc version not supported when it tries to compile CUDA kernel. So I run blender with:
CYCLES_CUDA_EXTRA_CFLAGS='-ccbin gcc-7' ./blender

This time Blender with GPU can dynamically compile the Kernel, and CUDA works when rendering with GPU only. But something strange happens, if I try to render with CPU+GPU, tiles corresponding to CPU are hung up and show only in black. If I press Esc key, GPU tile ends but CPU tiles are still hanging and black (Tasks manager shows CPU still in use) and I have to kill Blender process. Just mention that if I render on CPU only (not CPU+GPU) that problem does not occur, so GPU only and CPU only works correctly, but not CPU+GPU.

So, what would be the best solution for that gcc problem in my case? Is CPU+GPU problem also related to gcc version?

I tried with CUDA 10.1 from ubuntu repos and CUDA 10.2 from run file, the same problem with both
Edit 2:
To test having only gcc7 installed on my system, I cannot do it because when trying to uninstall gcc9/g+±9 related packages from my system, it also tries to uninstall other fundamental packages from the distro.

try passing -DCUDA_NVCC_FLAGS="-ccbin gcc-7" to your cmake invocation, it should perform the same thing but at build time rather than runtume.

Ok, thanks for the reply. I will investigate about how to do that.
Anyway I have tried what documentation says about “Remove compatibility checks”:

My file is in “/usr/local/cuda/include/crt/” and the related line is “#error – unsupported GNU version! gcc versions later than 8 are not supported!” (CUDA 10.2 installed from run file). I have removed that line and CUDA kernel is compiled with gcc9 without giving the error message. Cycles GPU only works correctly, but I still have the aforementioned problem about CPU+GPU rendering. CPU+GPU rendering is working correctly in buildbot builds. My CPU is i7-3770 and GPU a GTX 960.

About gcc version problem. Is there anything that can be done by Blender devs to avoid this compatibility error? Soon Ubuntu 20.04 will be released and I suspect that many users who want to build blender will have that problem.

We have an nvrtc based cuda compiler that is not dependent on gcc for this exact reason with MSVC where it used to be extremely picky on compiler versions.

But we have not used it in quite a while so not entirely sure if it still makes good kernels. You can enable it with the WITH_CYCLES_CUBIN_COMPILER cmake option.

Ok. I see. I still have the problem with CPU+GPU rendering.

I have done an experiment whose result seems to indicate that the CPU+GPU rendering problem I have is not related to CUDA compiler.
I have copied buildbot build cuda kernels (entire lib folder) from “/2.83/scripts/addons/cycles/” to the same path in my build.
I still have the problem running my build with CUDA kernels from buildbot builds (Runing builbot build works correctly).

Well, I have disabled WITH_CPU_SSE and now CPU+GPU render works on my builds made in Kubuntu 20.04.
What dependence on Kubuntu 20.04 could be causing CPU to enter some kind of infinite loop when rendering with CPU+GPU and build with WITH_CPU_SSE enabled?
The problem does not occur with Buildbot builds and neither portable builds built in Kubuntu 18.04 when I run it in Kubuntu 20.04. The problem only occurs with builds built in Kuibuntu 20.04.

Any clues about where to look to try to solve the CPU+GPU problem I am having in my builds when I compile with WITH_CPU_SSE enabled? Any software library or dependency? Maybe the Linux Kernel used in Ubuntu 20.04?

Well, that CPU problem when rendering with CPU+GPU is gone for a few weeks now.
But now I have these artifacts in my own builds with GPU render that I didn’t have before (it doesn’t happen with buildbot build):

The problem occurs in my own builds of both Kubuntu 20.04 and 18.04.
Maybe because CUDA 10.2 version? (Anyway, I was using that version before too and I didn’t have those GPU artifacts)

If I remove “/2.83/scripts/addons/cycles/lib/” folder from buildbot builds, then I get the same artifacts with GPU rendering when kernel is compiled on my machine. It must be something related to CUDA or compiler, but I’m not sure why some weeks ago this problem did not occur.

Cuda 10.2 is known to have issues with cycles on certain cards, see T74423 for details, no solution as of now, so downgrading back to 10.1 may be the best option while the problem is being worked on.

1 Like