[Discarted] GSoC 2022: Introduction and project idea - Script node for Geometry Nodes

EDIT 2: I talked to Hans Goudey and he told me that they’re not planning to make a script node for the time being. That means that this idea will not make to the GSoC. Thank you all for your interaction!

EDIT: Thank you all for the support. I’ll refine and correct some errors in my description so becomes clearer what I’m thinking about. Please read it again if you already have read it.

Introduction

I’m getting started in the Blender development community and intend to participate in this year’s Google Summer of Code.

I’m a Computer Science undergraduate from Brazil and I’ve been using blender since 2014. I already started to explore the codebase and also submitted a patch to this task: T46809

Since I’m still getting to know the codebase and and this community, my idea for a project might be redundant or unfeasible, so any feedback is more than welcome!

Project Idea

Overview:
Create a “Script Node” in Geometry Nodes and implement the necessary interface to enable running Python scripts with data from other nodes.

Node structure:
This node’s initial state would consist in only a field to select a script, similar to the Script Node from Shaders, with no input nor output sockets.
Captura de tela de 2022-04-11 10-31-56

The sockets would appear after a valid Python script is selected. Their names and types would be defined inside the script.

Captura de tela de 2022-04-11 11-17-26

Script structure:
The script should specify what are its expected inputs and what are its outputs. I’m still not sure about how to implement that, maybe I can use class inheritance and reflection.

Either way, the presence of these input and output descriptors would determine if the script is valid for the Script Node or not.

There would be an API that allows manipulating the data received as input. For example, there would be methods to transform geometry, create new vertices, etc.

Example:
The following script is an example of how this could work. To be valid, the script must have declared a class that inherits from a base class and has some pre-defined methods implemented:

# Made up module and types
from GeoNode import BaseClass, GeometryType 
import math

# Set the instances in a Fibonacci Spiral
class FibonacciSpiralInstances(BaseClass):
	def __init__(self):
		self.geometry: GeometryType = None
		self.angle: float = 0
		
	# override BaseClass method
	def getInputTypes(self)-> dict:
		return { "Instances": GeometryType, "Angle": float}

	# override BaseClass method
	def getOutputTypes(self)-> dict:
		return { "Instances": GeometryType}
	
	# override BaseClass method
	def setInput(self, nodeInput: dict):
		self.geometry = nodeInput["Instances"]
		self.angle = nodeInput["Angle"]

	# override BaseClass method
	def execute(self):
		idx = 0
		if(self.geometry is not None):
			for instance in self.geometry.instaces:
				fib = self.fibonacci(idx)
				xcoord = math.cos(self.angle * idx) * fib
				ycoord = math.sin(self.angle * idx) * fib

				# set instance position in local space
				instance.setPosition((xcoord,ycoord,0), local=True)

	# override BaseClass method
	def getOutput(self):
		return {"Instances": self.geometry}	
				
	
	# user defined method
	def fibonacci(n: int):
		a, b = 0, 1
		for i in range(0, n):
			a, b = b, a + b
		return a

Benefits:
This node would allow much more flexibility in creating procedural geometry. Although many things that could be done in the script already are achievable through existing nodes alone, the possibility to perform these operations repeatedly ou recursively opens the door for more creative power.

I know that the idea of allowing loops in nodes has already been discussed somewhere in this forum, and I remember that allowing such a thing would imply in a lot of complications about resolving circular node dependencies. The Script Node would allow repetition and recursion without introducing these problems to the node processing engine.

Challenges and limitations:
Two challenges that I can think of are: performance and how to expose the data and methods to the script.

I still don’t know if the nodes are written in C/C++ or Python. If it’s the former, copying data to the Python environment and running the script could be a performance bottleneck. Maybe it’s something that the user would’ve to be aware of.

About exposing data and methods from nodes in Python, if there’s already an API for that, that’s great! If not, implementing one can be an enormous task, maybe beyond the scope of GSoC.

19 Likes

Definitely missing the script node from grasshopper. As well as the expression/evaluate node.
(to avoid potential copyright problems, I’ve deleted the screen shots. )

1 Like

I think this is a great idea, but I think there are already a few people working independently on scripting for geometry nodes (@LukasTonne, @Wannes) though I’m not sure what state those are in currently.
Maybe ask around in the blender.chat channel for geometry nodes to see what the best course of action might be?

1 Like

I am not really sure about the feasibility of this as a GSoC project, if accepted at all. Using a script node like OSL in Cycles does not seem like a good idea, e.g. what is the entry point? how do you define properties?

Better would be to be able to register custom geometry node types, in the same way we currently allow to register custom nodes. Then defining inputs, outputs, and various properties can be done using the current infrastructure. That would imply to expose the execution function specific to geometry nodes in the registration code. If having registration at this level is possible, then this should be straightforward.

However, in order for this to be useful, one would need to also write APIs for accessing data the way geometry nodes work and expect (e.g. GeometrySet, the field evaluation system, etc.). That is a lot to chew on. Note that geometry nodes work on the entire geometry, so any existing API to work on a mesh or point cloud will work out of the box.

Then probably the RNA (used to bridge Blender to Python) would also need to be modified to allow generating C++ code, currently it is all in C (for developers passing by, I am talking about the various rna_*_gen.c that makesrna generates). This would be needed in order to expose the geometry node execution structures and functions to the RNA without having to wrap them all in C first.

I already have some of the building blocks implemented in a private branch to allow for defining modifiers in Python (here with custom nodes creating geometry, those are not GeometryNodes):

Even with those building blocks (that I could share), there would still be a lot to do.

Nodes are written in C++. However, accessing data from Python does not require a copy, the Blender object is simply wrapped into a Python object. The main performance bottlenecks will come from processing a potentially large amount of data.

As far as I know, those are for simple mathematical expressions. Useful for attributes and fields, but not so much for actual geometry processing.

4 Likes

Would the goal be to do write node trees with code or is it to expose all of the geo nodes data so that it can be used by addons in any way they like?

Ignoring the feasibility, I think there is value in both, and the former should be much easier.

I’m kind of thinking of a domain specific language that maps 1:1 to existing nodes. That way there wouldn’t be much of a performance cliff.

Something like

// made up nonsense syntax just to explain what im thinking of
(frame, direction, speed, geometry) = input;

offset = direction.normalized * speed;
frame * offset |> geometry.translate;

output = (geometry);

afaik the leading software in the node based stuff space has a DSL that is similar to C, kinda like glsl

such a language could also have macros, to support unrolled / copy-paste loops, without requiring a loop node to exist

such a node tree description language could be universal between editors (the only difference being what nodes are available) and should probably be designed in a way that it’s easy to generate code for it (for example add-ons in python could generate a script node instead of inserting lots of nodes (which requires laying them out in a useful way in the editor))

just my 2cents though

2 Likes

I hope that the script that I just provided in the post answers these questions.

That’s also one of my concerns…

I hope that my example illustrates an use case that can’t be accomplished with just nodes (at least not easily)

It would actually be to process geometry in a script, and this script be embedded in a node so it could be used in a node tree.

Hopefully my new example illustrates better what I meant.

Well, Blender already supports creating custom nodes, but not for builtin node tree types (shader, compositing, geometry, texture). For this, as I said, we would need a way to register this custom geometry node type.

You cannot use the __init__ method in Blender, neither can you reliably add slots to Python objects (i.e. no self.angle = 1.0). To reuse your example, with the Blender way, it would look like this:

import bpy
import math

# CustomGeometryNode needs to be invented.
from bpy.types import CustomGeometryNode

# Set the instances in a Fibonacci Spiral
class FibonacciSpiralInstances(CustomGeometryNode):
    bl_idname: "FibonacciSpiralInstances"

    # Define a float property, this will not be a socket, but button in the node
    angle: bpy.props.FloatProperty(name = "Angle", default = 1.0)

    # This is not the same as __init__, but the way Blender initializes nodes
    # __init__ is not really used in Blender
    # Here you would setup the inputs and outputs.
    def init(self, context):
        self.inputs.new('NodeSocketGeometry', name = "Instances")
        self.outputs.new('NodeSocketGeometry', name = "Instances")

    # override BaseClass method
    def execute(self, exec_params):
        # These would directly map to the C++ API.
        geometry_set = exec_params.get_input_geometry(0)
        instances = geometry_set.get_instance_component()

        if not instances:
            exec_params.set_default_output_values()
            return

        idx = 0
        for instance in instances:
            fib = self.fibonacci(idx)
            xcoord = math.cos(self.angle * idx) * fib
            ycoord = math.sin(self.angle * idx) * fib

            # set instance position in local space
            instance.setPosition((xcoord,ycoord,0), local=True)
            idx += 1


    # Draw the buttons, this is optional.
    def draw_buttons(self, context, layout):
        # Draw the angle property in the node
        layout.prop(self, "angle")


    # user defined method
    def fibonacci(n: int):
        a, b = 0, 1
        for i in range(0, n):
            a, b = b, a + b
        return a


# Register this node type into Blender's NodeType database
# Typically called when enabling the addon.
def register():
    bpy.utils.register_class(FibonacciSpiralInstances)


# Remove this node type from Blender's NodeType database
# Typically called when disabling the addon.
def unregister():
    bpy.utils.unregister_class(FibonacciSpiralInstances)


# If the script is ran from Blender's text editor
if __name__ == "__main__":
    register()
1 Like

Thank you for your insight! I didn’t know that Blender had support for writing custom nodes.

Considering all of this, would it be feasible to use the existing custom node API to create the “Script Node” that I’m proposing?

Using the custom node system to define custom geometry node should be possible. As said in my first post, the main work would be to define APIs for all the specific geometry node processing, and potentially add more APIs for general geometry processing (Mesh, PointCloud, etc.).

I would suggest to talk about this in the Blender chat with the geometry nodes team, there is a link to it in some earlier post. I cannot say if it is acceptable, I only speak as a Blender developer on the feasibility of it.

To get familiar with how to create custom nodes, in the Blender Text Editor, there is a “Template” menu where you will find a “Custom Node” script, that can be run, tested, modified, etc.