Hello,
I’m interested in knowing how I can use Nvidia’s CUDA (which is primarily written in C/C++) in my Blender addon.
Here’s what I would like to know:
Is there a community standard/recommended way to handle integrating C++ code into Python? Or more specifically, CUDA code in Python addons?
There’s a binding library called pybind11 which can let me call/run my C++ code from within Python. How do I property package external libraries and dependencies such as pybind11 if I were to publish my addon?
How would my C++ code be packaged with my Python addon if I were to publish it?
Here’s what I already know:
I can’t do a pure C/C++ addon. The part of the addon that interfaces with Blender must be done in Python, this is non-negotiable.
There is a Python wrapping/version of CUDA, but I do not want to use this because I would like to utilize the full performance of C++ (as much as I would love to use Python).
I’m interested in building a render engine (I know, I know, I’ve scoped it down. It’s for educational purposes). My engine will run exclusively on CUDA C/C++ with Python only there for interfacing with Blender.
I’ve made progress with implementing pybind11, so now my current issue (assuming this is the recommended route) is running python code in Blender. My python code can’t find the C++ module created by pybind11 when it is ran using blender -b -P codeFile.py, but it works fine from the standard interpreter (python3 codeFile.py)if I launch it directly from the bundled python environment /snap/blender/current/2.91/python/bin/python3.7m codeFile.py (I’m on Linux btw).
Please let me know if there’s anything else I can supply to help.
Ahh, OK. Cycles might be a great thing to look at. As far as I understand it, Cycles is doing exactly what you want to do – with the exception that it’s pulling some of its data through a shadowy C quasi-API. LuxCore does all of its data transfer through Python IIRC.
No, I don’t think you should attempt to use a single Python file. Create an addon, put the C++ library in the addon. Then it should be accessible with import addonname.some_folder.some_module or whatever. Addons are located in ~/.config/blender/2.xx/scripts/addons.
Ok… I think I understand your suggestion. Now, my current issue is I can’t get the import addonname.some_folder.some_module part to work.
Through pybind11, I have a python module in the form of a binary located in the build directory of my C++ project. I then use sys.path.insert(1, path/to/my/build/dir) to include the package into my Python path before calling import MyModule. If I run this with the normal Python interpreter (python3 codeFile.py), it works. However, the interpreter in blender (blender -b -P) doesn’t work.
How can I fix this?
(Note: I’m correcting my last post. I can’t seem to reproduce getting my code to work with the blender interpreter at /snap/blender/current/2.91/python/bin/python3.7m)
I think that’s just the wrong way to do it Use an addon. And you’ll want to copy all the files into the addon. You can’t really trust the paths to be where you expect them, and the path functions are also different depending on the OS. It’s best to avoid using them altogether if you can.
Anyhow, this is the way I do it when I need to:
# Thanks Russel Dias, Mark Amery
# https://stackoverflow.com/questions/5137497/find-current-directory-and-files-directory
import os
dir_path = os.path.dirname(os.path.realpath(__file__))
And I think you have to use \ instead of / on Windows paths… don’t remember.
Besides this, I donno too much. I’ve gotten a C++ library (curve_fit_nd) to import but I’ve never really used it yet (portability concerns).
Hmm… I think I may have a fundamental misunderstanding of what an addon is… I’m new to this stuff. Based on this tutorial from blender, unless I completely misunderstood it, I’m supposed to run create addons as a script, test it via blender -b -P or the internal script editor, and then install it by placing the (single Python file) into the addons folder through button in the addons section of settings.
How do I get a binary file/library/module to come along in this process? (sorry if this is a trivial question)
Alright, made more progress and am now at this problem:
pybind11 is using my built in Python interpreter (3.8) which does not match the one in Blender (3.7). Now, I’ve tried setting what interpreter to use (as per their instructions) with
But when I reload the addon in blender, it still says that the module (which it can now see, yay) was built with 3.8.
Any ideas on how I can explicitly set the Python interpreter to the one bundled with Blender with the CMakeLists.txt? I’d prefer to use the bundled interpreter rather than going out to install Python 3.7 onto my computer.
Bearing in mind that I am not a C++ programmer of any merit, it sounds like an ABI incompatibility problem. So maybe it’s the C++ module that’s being compiled against the wrong Python? That’s my guess.
@Microbob sorry for interfering, but while Cycles is a good example, it’s also very integrated into the Blender code itself, so looking at a different render engine that is also an addon may be a better idea.
You can take a look at LuxCore Render, it’s a magnificient render engine and it’s very well developer, both the engine and the addon, it may give you some ideas.
I used the submodule approach… so I didn’t exactly build pybind11. I did use the CMAKE argument to explicitly point pybind11 to use a python interpreter matching the one blender uses (Python3.7m).
This approach with pybind has become really messy. CUDA wasn’t playing well, and there are a lot of project files. I’ve moved on to use sockets (for better or worse) and am now try the approach where I have a central python server launching and communicating with compiled (C++) executables. I’m going to leave this this post open though in hopes that someone else may come by with a better solution. Current issue I’m tackling here is relaying large amounts of data over a socket connection (converting floats in C/C++ to string is slow)
Use an addon. And you’ll want to copy all the files into the addon. You can’t really trust the paths to be where you expect them, and the path functions are also different depending on the OS. It’s best to avoid using them altogether if you can.
. Hopefully this will be enough to help you 
