Custom Python Nodes

Hi,

I have been using blender as an artist for years. Recently I got interested in building addons for it. I know the basics of python so I am able to make simple operators, UI elements, etc. mostly for automation.

Now my question, I am currently learning how to make a custom node tree. The goal is to use the nodes to write data to external file (XML). I understand the project is similar to animation nodes and sverchok but nowhere close to their complexity.

I have tried using the template that ships with blender to make a custom node tree with nodes and sockets. I am also able to change the buttons/labels on the nodes. But I am faced with this big road block right now - I am unable to execute anything with these nodes.

There is almost no documentation on this topic. I know the nodes are just an UI and I have to build the backend myself but there’s no hint on where to start. I want to combine python code with the node data. For a simple example:

Suppose a node has a property my_float. Now I want to write the value of my_float to a XML file. Then how can I do that?

Right now I am unable to access property of a custom Node class from outside of that class. Also I am unable to put any execute() function inside the Node class either. So I am stuck without any tutorials or advice.

I have looked everywhere I could so far, including the animation nodes source code. But it is not very helpful right now. I do not understand all the concepts used there and couldn’t figure out how the nodes are implemented. A bit overwhelming for where I am at currently as a developer.

I know I can reach a lot of experienced developers here and also the core devs who made the api. Any help is immensely appreciated - if there’s any documentation on this that I missed then please point me to that too.

I hope my post is not too lengthy. I wanted to make it very clear what exactly I want to achieve.

Thanks in advance.

Regards,
-Sayan.

1 Like

Since documentation is so scarce on this topic, I would like to request @jacqueslucke to chime in and shed some light on this problem. Knowing your work on animation nodes, any contribution/advice coming from your side will be highly valued.

Regards,
-Sayan.

1 Like

Blender’s nodes are basically a UI feature. You have to write a lot of the code yourself. I was working on a node-tree system, for example, and I wrote a lot of code for walking the tree to find out the current value at one point or another. It’s not very easy. I also copied a fair bit of code from Sorcar and a little from LuxCore. There are a lot of node-based addons around, it’s worth checking a lot of them so you can see how multiple people do the work.

More specifically: I have a Python file defining classes for nodes and node sockets. I have another file defining operator classes-- in my case, “Execute Node Tree” and “Query Node Values” (this is a utility function for checking if things work). The Execute Node Tree function has a file defining the correct way to walk the tree, read the data, and build the hierarchy of “actions”. There is one more file that defines the behavior of these “actions” - one class for each node (the “action” class is a more generic version of the node, with no bpy code except for the execute function). I’ve separated the “actions” from the nodes themselves, because they need to be processed separately, and because my goal is to have a generic interface that I can use to run the actions – the Node Tree is only one user input method, I’d like to have file I/O from a kind of simple markup language, too. Nodes are easier to use for testing, though! So the Node tree is user input. Each node has several sockets which inherit or set values of various types. The connections between the nodes represent inheritance of values. And the tree itself represents a series of actions the tree-interpreter must perform in order.

My advice is to start by focusing on the UI. Think about the way it will work from the user’s perspective. How does the data move around? Get a good design on paper, then start trying to write the code.

why not to use:
http://nikitron.cc.ua/sverch/html/nodes/text/text_out_mk2.html
Node to make xml (CSV) also json there

First, thank you so much for the elaborate answer. This is the only place I got any information on this topic so far :slight_smile:

I completely forgot about Sorcar and LuxCore, I just checked their repo - it seems to be more comprehensible at first glance. If you have any other similar projects in mind, a list would be awesome. I would love to learn from all of them. (Including your own project, in case the license allows it and you are okay with it of course)

I understand, totally agreed that I need a solid design first. I realized that too, as I started working on this project. I wouldn’t say it is complete but I need a basic understanding of data flow first to finish the design.

I got a lot of clarity from your answer, but I still have one confusion. How do you access the user input from the nodes? I assume the input is taken as properties (bpy.props?) but I haven’t been able to access them from outside the class? Or is it my lack of python knowledge?

This very specific issue is proving to be a big road block for me. If you can help clear this matter then I think I can figure out the rest with a lot of work and practice.

Regards,
-Sayan.

Stoked to get an answer from you. I tried finding you here, but the username is different from that of Github (an anagram as I now realize)

I think this is almost exactly the kind of node I am trying to make. But I am trying to learn the development process to make my own tools similarly later on. So I am trying to understand the node source code now, from the repo.

Thank you this is very helpful. I do have a few follow up questions (maybe novice qns):

I see there are more functions in the node class than available in the “custom template” that comes with Blender. Also it is a subclass of “node” class instead of “Node”. My understanding is “node” is a wrapper class/extended class from “Node” and you are not using the actual “Node” class at all?

Am I thinking in the right direction? Then again my biggest hurdle so far, how do you access the properties defined in the node class from outside the node.

More clarification is really appreciated. Already got a lot of hints from the source code. Thanks again for your resource and time!

Hey, happy to help :smiley: I use Gitlab, but I haven’t published my node project yet, it isn’t useful yet and I want to get a lot more stuff working before I let anyone see it. I’m ashamed of how messy the code is right now :cry:

I don’t have a list of node-based projects on-hand, the only one I can think of that hasn’t been mentioned here is Modular-Tree, although its simple node-tree may be worth looking at.

How do you access the user input from the nodes? I assume the input is taken as properties (bpy.props?) but I haven’t been able to access them from outside the class?

Ah, well in the first place you should be working with instances of the class instead of the class itself. And as you mentioned earlier, the built-in classes aren’t all that useful unless you extend them in your own classes. Anyhow, it’s not much good to get the value of a class property! The node sockets are where the properties are stored, and you have to define these values as bpy.props values (the convention is to name this property “default_value”). You can define whatever functions for the class you may need, and a “tell_value()” function can be helpful. There are a few built-in callback functions (including one that is unimplemented.) Sockets are themselves instantiated in the nodes’ init function (don’t confuse this with __init__ which you usually can’t define for extensions of built-in classes).

Anyways, there’s one very annoying difficulty when it comes to getting from nodes to values: sockets are on the right (input) or left (output)… not both! So when I wrote a function for seeking back through the tree, I had to write a function for each node that says how to travel “through the node”, that is, from the output socket of the node to the corresponding input socket (data can travel through several nodes before reaching its destination, and it can be modified along the way, e.g. with a math node or something like that). So the function went something like this pseudocode:

socket = some socket
while(socket):
  if socket in node.outputs: #this isn't a real property of node, but there's something like this IIRC
    socket_b == node.traverse_socket(socket)
    # node.traverse_socket is a custom function I have defined for all my nodes
    if socket_b is None:
      #(we've reached the end, so return the current socket)
      break
    else:
      socket = socket_b
      continue
    
  if the socket in node.inputs:
    if socket.connected == True:
      connection = socket.from_connection
      if connection.from_socket:
        socket = connection.from_socket
        continue
      else:
        break

Something like the above will eventually get you the first socket in the tree that is sending data into the tree by a value the user has set or by a node that provides data. Now you have to get the data from the socket, and if you define a default_prop for each socket, thats easy enough. Another option is to define a tell_value() function for each socket. Finally, the node itself may be what is relevant, in which case it’s still up to you to give it the appropriate property or function.

I use this function for going backward through the tree to query values. This is a good way to work when your node graph has an output node that gathers input from multiple input nodes.

As it turns out, my node-tree is a little backwards… it has a single root node that has a lot of children and those children can have children, so it is very much structured like a tree. So I wrote a very clever function for walking through the tree and returning every traversable line through the tree. This was very hard to write, and I won’t attempt to render it here as pseudo-text; it was essentially a Python version of this:


(see Tree traversal - Wikipedia)

I’d love to share my code with you if it will help, but I will need to do it in a paste-bin or something since it’s not ready to publish yet! I want to have a minimum viable product and a lot more error handling done before I do that!

Honestly, this is one of the more difficult parts of the Python API that I’ve used. It was very challenging for me to get as far as I did! And I’ve been doing this a little while. (You might be wondering why I haven’t finished it – I don’t get much free time for this and I have been focusing it on improving my Rigging Addon… I think I can have it ready for 1.0 by the end of the year :relaxed:).

I understand that this is probably a little difficult to follow – it’s hard to write about this in the abstract. It’s also not written very clearly (it’s 1 AM now). So let me know if you need me to explain anything further!

EDIT: In fact, one big thing that I forgot is this: most of the time you will use bpy.props that you define in the node to send data into the tree, only unconnected input sockets are really useful for giving input from a socket. Finding the “first” socket, as I explained above, may be useful simply for finding the first input node. The node and the socket each have draw functions for drawing their properties (just like operators) and you can draw a property in the UI and modify it and query it. That’s a good way of getting information into the tree. You can also have nodes that initialize the values at creation or use update callbacks if you want to get fancy.

1 Like

That’s dense! I understand a few things and clueless about the rest, haha :sweat_smile: but I believe I can figure things out. I hope you don’t mind me bugging you with too many questions!

Thanks for the Modular-Tree reference, it seems simple and useful. And I totally understand that you want to refactor your code first before unveiling it. I also checked out your rigging addon, the idea is promising! Will be waiting for the release.

Like I said, there are lot of things that I do not fully understand yet but if you elaborate on these following doubts I have then I can make more progress:

About using the instances, do you mean I should iterate through the node tree using a different function?

Your edit paragraph most interests me at this point. Indeed the one node that I have made for my project takes input though a draw function. It uses a StringProperty to store the value. But how do I query it? I am stuck there.

Thanks for your time, I appreciate your assistance. It is the only way I am able to progress so far on this project :slight_smile:

EDIT : I just figured out those above questions myself. It was silly now that I see it haha :sweat_smile: I’m making good progress now. But I might need assistance for the final tree execution code (to traverse the node tree). Thanks for all the help so far!

1 Like

Only one sidenote here: I am considering if tree traversal can cause slowdown of the entire graph at some point in time. If there are two many nodes in the graph doing too many traversals. The only way to gain speed is to create a global context and just let nodes write their stuff there.

I am interested to hear if any of you run into this sort of problem - if you ever considered it.

Hello,
maybe off topic, not related to nodes tree traversal so much; but more to evaluating Python using input and output sockets in a new type of node. There is an open-source patch proposing 3 custom nodes; that has not been integrated into Blender. One of them allows any Python code especially using numpy for image buffers processing, to be evaluated: https://github.com/bitsawer/blender-custom-nodes
The nodes code are also simpler to study in the project’s patch.
Best

2 Likes

Yeah, I am worried about that too. I don’t know if I can figure that part out. Currently working on the other parts of the project. Surely need to experiment with a few designs before I get something decent. Also, nice to see you here! :slight_smile:

1 Like

I took a look at that, since it is a patch I don’t think I can apply the knowledge directly at the moment. Anyway thanks for the link, I am grateful for any resource I can get my hands on and learn from!

It’s been a little while since I’ve touched it but I have some functions on hand for going backward and forward through a node tree. Give me a shout if you want to look at them!

This may be of some help to you:

There is a thread on here about it, detailing. progress and some of the snags I encountered along the way:

It’s a Music & Midi node system, written with its own node editor and complete structure with menus, resources, etc. It may be more complicated than you need, but it will show you how to build such a system. It is still WIP, so there is little code documentation, docstrings, etc and definitely no help files yet… Anyway, it might be helpful and I am happy to answer any questions you might have.

It is the first node system I have built so might not be perfect, but it all works. One thing it does show is how to pass multiple variables through node trees. I use dictionaries for this and pass them through only one output socket per node. I found this to be the easiest and more reliable method. I have also done quite a lot of development with AN, you can find that on my GitHub too, but in the end AN became too big for what I wanted, so I set about writing my own node editor. You will see that I can run this either one frame change, or via a timer, that bit might also help you get your tree to execute.

If this is useful and you have questions, just fire away!

Cheers, Clock. :tumbler_glass:

3 Likes

Thanks @clockmender. I find that this area of Blender’s Python API is sorely lacking in quality documentation, largely because it leaves so much up to the addon developer to implement. When I have some time later I’ll read through it and give you my input :slight_smile:

1 Like

http://nortikin.github.io/sverchok/docs/nodes/text/text_out_mk2.html

1 Like