Would it be possible to do import bpy without initializing CUDA drivers?


#1

I recently ran into an issue when trying to move my data to GPU using PyTorch’s Python API. By reading a couple of threads[1][2][3] I noticed that CUDA is not fork-safe. The only way the problem can get resolved is by not calling any CUDA driver before calling a forked process (you can do whatever you want in the process though). After trials and errors I realized that doing import bpy is causing this issue for me as I guess the import process is calling the CUDA drivers somewhere while getting loaded. So I wonder is there a way that I can disable this? I don’t want import bpy to call the CUDA drivers before I ask for it. FYI, I have compiled Blender as Python module and have been using it in my machine’s Python. It would also be okay for me if I have to compile Blender without letting it access to CUDA drivers. So please advice.

Here is a comment from someone who had a better idea of what’s exactly happening behind the scene:

CUDA, as a complex, multithreaded set of libraries, is totally and permanently incompatible with a fork() not immediately followed by exec(). That means the multiprocessing method fork cannot work, unless the fork is done before CUDA is initialized (by direct or indirect call to cuInit()). Once a process goes multithreaded or initializes CUDA, it’s usually too late.Second, torch.cuda.manual_seed() is lazy, meaning it will not initialize CUDA if it hasn’t been done already. That’s a good thing, for the reasons above.

Btw, I’m not entirely sure but this might also be relevant to this bug or this bug that I reported earlier this year.

Update:
I posted a bug report for this: https://developer.blender.org/T57813


#2

It looks like a recent commit has fixed this :slight_smile: