I recently ran into an issue when trying to move my data to GPU using PyTorch’s Python API. By reading a couple of threads[1][2][3] I noticed that CUDA is not fork-safe. The only way the problem can get resolved is by not calling any CUDA driver before calling a fork
ed process (you can do whatever you want in the process though). After trials and errors I realized that doing import bpy
is causing this issue for me as I guess the import process is calling the CUDA drivers somewhere while getting loaded. So I wonder is there a way that I can disable this? I don’t want import bpy
to call the CUDA drivers before I ask for it. FYI, I have compiled Blender as Python module and have been using it in my machine’s Python. It would also be okay for me if I have to compile Blender without letting it access to CUDA drivers. So please advice.
Here is a comment from someone who had a better idea of what’s exactly happening behind the scene:
CUDA, as a complex, multithreaded set of libraries, is totally and permanently incompatible with a
fork()
not immediately followed byexec()
. That means themultiprocessing
methodfork
cannot work, unless thefork
is done before CUDA is initialized (by direct or indirect call tocuInit()
). Once a process goes multithreaded or initializes CUDA, it’s usually too late.Second,torch.cuda.manual_seed()
is lazy, meaning it will not initialize CUDA if it hasn’t been done already. That’s a good thing, for the reasons above.
Btw, I’m not entirely sure but this might also be relevant to this bug or this bug that I reported earlier this year.
Update:
I posted a bug report for this: https://developer.blender.org/T57813