Hello! I’ve recently stumbled upon an issue while developing an add-on and spent a few hours trying to piece together some answers from Blender’s source code without much success. I found very little documentation about this topic.
Would it be possible for someone to provide a high level architectural overview of Blender’s Python API that could be composed into a document to go in https://wiki.blender.org/wiki/Source?
Here are some topics that I think should be featured:
How are Blender objects, allocated from C, reflected as Python objects?
How does the DNA/RNA system contribute to expose data and functionality to Python?
How/when does Python run from within Blender? (add-on registration, startup folder, UI python files, text editor scripts, Blender’s Python console)
Threading considerations: are add-ons and scripts always run from the main thread? How about bpy.app.timer, handler and operator modal callbacks? When is it not safe to read or write to Blender objects?
How does bgl (and anything else that Python causes to render) fit into the normal render pipeline?
What are the relevant bits of code to look for?
Are there any known design limitations or future plans?
Are there goals for the API? For example, in my experience, Blender’s Python API stands out very positively for “traceback errors instead of crashing” or “all blender data and operators are accessible through Python”. This is very much by design, as official UI uses the same system and therefore it is impossible to “forget to expose something for scripting”. I think it’s worth mentioning what is working well.
I am available to help structuring the document, with the phrasing or provide diagrams, but I don’t have enough understanding to provide the core content.
The issue with my add-on is already fixed, but the request for a high level document is still valid.
I was creating a sequencer strip from python, keeping that variable, and using it to update the strip’s data in a modal operator TIMER event.
If the strip gets deleted by users, the variable is never None, but its contents may or may not have been cleared, which leads me to think that there is a race condition and, therefore, an operator’s modal does not run on the main thread.
For me the specific case is that when I changed scene objects from bpy.data, I would spend many minutes to find why code didn’t work, but when I actually switched to 3D view and performed some type of event (mouse move) things would be evaluated. More or less this was a sign that I would need to force update the dependency graph update. As the RNA is a tree structure and is evaluated lazily (think of the “dirty pattern”) usually you will want to do it yourself so you can have immediate feedback on the state of things.
I moved this to the Blender Development part of the site. Although this is Python related, this is a core development question.
The “Other Topics” category (and its subcategories, such as Python API) are self-managed by the community. Meaning they get less attention from the core developers.
@brita these are good questions…
I’ve found with API’s there often ends up being a lot of unwritten knowledge that’s gradually gained through experience - however there is no reason needs to be this way - these questions can have plainly written answers.
The more general questions don’t fit so well in existing documentation… so we could do the following.
Gather some more questions regarding the Python API.
Write answers to these questions (I’m fine to handle some of these + contributions/feedback from others too).
For each question either add the answer inline, or link to an external page if this information makes more sense to add elsewhere.
Note that some of your questions are a bit more fuzzy:
“Are there goals for the API?” (the answer for this is likely to be fuzzy too, you further detail these question though. Those could be covered by an FAQ).
“Are there any known design limitations or future plans?” (regarding future plans, not so relevant for an FAQ as it’s too time sensitive).
Some of the questions would benefit from being answered directly in the API docs, either as an increment to already existing places or, as you mention in FAQ format.
However, I listed those questions, specially the fuzzy ones, from the point of view of trying to understand the core design and current implementation of the Python API (as opposed to the point of view of an API user). That’s why I was suggesting a document in the development wiki. I was thinking of something more like the Cycles architecture docs: https://wiki.blender.org/wiki/Source/Render/Cycles.
This would be a place that would guide developers extending or fixing the API.
In either case, the first step would be to start writing down knowledge in whatever format is easier. Then, we can structure it and fit it where it’s helpful. @ideasman42, if you could start documenting the fundamentals of the API design and implementation, I can read through, see if I understand and come up with follow up questions.