Code Generation with Python

We have the goal to reduce the number of files that need to be edited when adding a node. This is part of the more general goal of improving Blender’s extensibility.

Currently, the minimum number of files that need to be edited when adding a node is 7. I’m currently working on a patch that reduces this number to 5 by automating the node discovery process.

It seems like there is no other reliable way to do this other than code generation. Fortunately, the generated code is very simple. It just calls the register functions one after the other.

Now to the main topic: Currently the code generation script (discover_nodes.py) is written in Python. The main reason is that it is the easiest language for the task at hand. The problem is that Blender technically supports building without Python, which might be a problem if a build script is in Python.

I originally planned to move the code to C++ in the end, but @LazyDodo suggested that it might be better to actually use Python here. The main reasons are that it is simpler and avoids bad dependencies on other parts of Blender.

  • Do other developers think that Python is ok here?
  • What is the purpose of the WITH_PYTHON cmake option? Is it there so that Blender can be build without Python or so that one can build a Blender that does not bundle Python?
4 Likes

I don’t mind Python being a dependency for the build process.

It seems @ideasman42 added WITH_PYTHON, but I’m not sure what the reasoning behind it was.

I think this was at a time when Python was only used for extensions, so seeing it as just another optional library made more sense then.

I do not mind making python a building requirement either. Blender without python is (very) close to useless anyway…

We should probably remove the WITH_PYTHON option though, in that case?

For the case that we decide to remove the WITH_PYTHON option, I prepared a PR: #110793 - Build: remove WITH_PYTHON option - blender - Blender Projects

If someone has good reasons to keep WITH_PYTHON that’s fine too, but would be nice to have a better description for what the option means and who it is for.

I don’t mind, like MakeRNA, having the same modules - C++ programs for auto-linking nodes.

Using Python seems OK (I’m not strongly against it at least), although I think it may be best to use C++ long term.

Reasonably often I’m unable to run the Python binary distributed in: ../lib/linux_x86_64_glibc_228 there is a binary incompatibility that bites me from time to time. I couldn’t say how common this is though, I would expect to have heard others mention this if other developers were running into it often.
Whatever the case, not being able to use Blender’s built-in Python means developers may be using their system Python instead of whichever version we’re using. Which means the Python version is not guaranteed.

If the script is simple enough - we’ll likely get away with build-tools depending on Python, however if it’s more involved it seems likely larger upgrades (e.g. Python 4.0) could trip us up.

Whatever the case, moving to C++ is always an option so we can do so if the need arises.


Regarding WITH_PYTHON don’t think this poses much of an issue? We can have a find_program(PYTHON_EXECUTABLE ...) in an else block, or in the case of WIN32, always set PYTHON_EXECUTABLE point to the executable in ../lib/.

Ok thanks, than I’ll go ahead with my auto-registration patch now.

My main problem with it is that it is not clear what it is supposed to mean exactly. Like, under what circumstances would one disable it, and what is still expected to work when it is disabled? A comment for that in the source code would be useful when we don’t want to remove it.

Yes, this should be better communicated, and the use cases for this are quite niche (replied to the PR to remove it).

1 Like