Expandable Geometry Socket [Proposal]

This is a refined proposal on how to handle attributes sockets. It is a streamlined version of the Attributes Sockets proposal with the following differences:

  • No hard-coded names are expected/required, not even for local attributes.
  • Nodes that output a geometry and new attributes can have its attributes accessed directly as lists, and/or appended to the outputted geometry.
  • The geometry socket can be expanded to get or pop its individual attributes.

The first two parts (“Attributes and Attributes Sockets” and “Geometry Nodes Modifier”) are copied from the original proposal.

Attributes and Attribute Sockets

Attributes are commonly referred to as columns. This is a reference to how they are visualized in the spreadsheet. Attribute sockets are sockets that pass an entire column from the spreadsheet containing:

  • List (array) of elements.
  • Datatype (float, integer, boolean).
  • Domain (vertex, face, …).

So far every node were allowed to read and write any attribute to the geometry, and the result was incorporated as part of the outputted geometry. This leads to an extremely linear nodetree that makes it hard to read.

Instead, it will be interesting to pass the attributes around directly, so the operations can happen independently of the original geometry.

This was avoided in the original design because users can easily shoot themselves in the foot by connecting attributes from geometries with different index orders or length leading to unpredictable results. At the moment though, its benefits outweigh those issues.

Selections are just a an attribute socket with a list of booleans.

Geometry Nodes Modifier

Let’s start by looking at a simple geometry node modifier. The first new thing you can see is that all the
attributes expected from this mesh and generated from the node tree are explicitly visible in the modifier.

Note also that those attribute fields (mushroom, scale, rusty_weight, mushroom_uv) are not simply strings. They are attributes that are expected to exist in the mesh. While they are referred to as strings, inside the nodetree they are passed around as attributes (i.e., list of values).

image

Geometry and Attribute Inputs

Geometry Socket can have multiple Attributes sockets accessed from it. Those are the attributes exposed outside the modifier and mapped to UV maps, vertex groups, … The non-exposed attributes (e.g., other UV maps) are still preserved with the geometry and interpolated during topology changes.

Those exposed attributes are part of the geometry socket and passed along with it. However they can also be accessed directly. In this example the Mushrooms attribute is a vertex group mask defined in the modifier that is multiplied by two. It is then passed to the Point distribute node.

When the nodegroup is used inside another nodetree attributes can also be passed individually as a list, detached from any geometry socket.

Expanded Geometry Socket

At any point a geometry socket can be expanded to explicitly expose one of its accessible attributes:

This allows those attributes to be passed around as a list directly from its current values (not the ones from the Group Input). This is particularly useful when the geometry went through topology changes.

image

The Mushrooms list accessed after the Subdivide node has more elements than the one obtained directly from the Group Input. It can be used with the Point Distribute node for the subdivided geometry.

Geometry Expand Node

A “Geometry Expand” node should support the same socket expand operations. Similarly, a “Geometry Compose” node should allow the existing attributes to be replaced by different lists, besides adding new attributes to a geometry.

image

Attributes Outputs

New outputs created in a node are explicitly exposed as list outputs in the node. By default they are not passed with the geometry.

image

However, if an attribute needs to be accessed later, after the topology of the geometry changed, its sockets can be added to the geometry.

To simplify adding attributes to the geometry and access them, this can be automated when dragging using an extra keyboard modifier (ctrl / alt / shift):

geometry-nodes-connections

Attribute Domains

When converting from an attribute to a list it is important to indicate which domain this is intended to be used with. The value can then be interpolated from its original domain if needed.

image

This applies for both input and output attribute sockets. They were omitted in the other images in this proposal to keep things simple. But every attribute socket needs to have its domain identified.

Attribute Outputs and Local Attributes

Local attributes are attributes that were not in the original input geometry. They are accessible inside the entire node tree. However they are not automatically outputted with the geometry outside the node tree.

image

They need to be explicitly picked to export — in this following example Extrude: Side Faces is not outputted with the geometry (see the hidden icon). The outputted attributes need to be mapped in the modifier to an existent or new attribute.

UI/UX Design Challenges

  • What does an expanded socket look like?
  • How to expand a geometry socket to get or pop an attribute?
  • Different types of attributes should be visible differently in the spreadsheet and in the node tree.
  • Attributes sockets have an explicit domain for input and output that needs to be visible.
  • Lists sockets (e.g., attribute sockets) should be different than their regular counterparts of the same data type (float, integer, …).
  • Attributes outputted from nodes can be “added” to the geometry, need a visual language for this.
  • Attributes expanded from a geometry socket can be popped or stay in the socket, need a visual language for this.
  • The modifier and node user interfaces will need a new way to organize its buttons to allow for geometry attributes to be intercalated with other inputs.
  • Object info will need to change to be treated as an Input too, so users can map the Object as well as the expected attributes to operate on (at least when the object is defined in the modifier, not inside the nodetree).

Related posts:

20 Likes

For me the most interesting thing since the rise of the idea “everthing nodes”: Will i be able to use the voronoi texture node (the swiss army node from the shader editor) inside the GN tree and reuse it with its coordinate system inside the shader later? Or in the opposite direction? I always found the current texture editor limiting (try to change its voronoi vectors by a musgrave texture in “object” tex coordinates - as you easily can in a shader tree).

3 Likes

This is amazing! Maybe the design could get refined.
Eg: in complex systems the explicit socket listing could get quite overwhelming. So over maybe, over a limit the property sockets could collapse in to a compound socket that is the same under the hood as the listed out ones.
In addition to this we could add a new property for the sockets that specifies the thickens of the socket connections to make it a more intuitive and less visually noisy design.

UI/UX Design Challenges

  • What does an expanded socket look like?
  • How to expand a geometry socket to get or pop an attribute?
  • Different types of attributes should be visible differently in the spreadsheet and in the node tree.
  • Attributes sockets have an explicit domain for input and output that needs to be visible.
  • Lists sockets (e.g., attribute sockets) should be different than their regular counterparts of the same data type (float, integer, …).
  • Attributes outputted from nodes can be “added” to the geometry, need a visual language for this.
  • Attributes expanded from a geometry socket can be popped or stay in the socket, need a visual language for this.
  • The modifier and node user interfaces will need a new way to organize its buttons to allow for geometry attributes to be intercalated with other inputs.
  • Object info will need to change to be treated as an Input too, so users can map the Object as well as the expected attributes to operate on (at least when the object is defined in the modifier, not inside the nodetree).

One other problem that needs to be solved is attribute renaming. Currently you have to manually change the names of local attributes everywhere they are used, instead the attribute names should update automatically (just like when renaming an Object will effect Object input). This should however be optional and only effect downstream of the node tree. (this is low priority request)

Is pop short for populate here? I’d probably lean towards using Set instead. Get / Set is already an established and widely used naming convention.

What’s the benefit of separating Built-In, Input and Local? Are local just attributes set inside the node tree itself?

I think this is an exciting idea. I can imagine cases where you want to be manipulating data while also keeping the list length etc responding to topology changes so processing it in parallel as an attribute from earlier in the tree than you might want it means it can interpolate values through topological operations.

Is this node necessary? Can’t all geometry sockets be expanded? If not, how would they be differentiated to the user?
Geometry Compose node get’s a big +1 from me!

Really cool idea. Some more clarity to the design like maybe a different socket shape when it’s enabled. Just thinking of hard of sight / zoomed out clarity.

Isn’t it also important here to illustrate the number of elements in the list? For example, you might pull face normals from a plane object and use them as tangents for a set of curves starting at the face centres. In this case you might have 36 verts on the plane but only 25 face normals. You also have 25 face centres and it’s knowing the length of the data that’s important, so your 25 new curve objects can be created. When we start pulling attribute columns between disparate geometries knowing the list length is going to be really useful so we can manipulate the data accordingly.
(In Sverchok the sockets only display the number of objects coming through when connected and it shows on both input and output sockets which has saved me a lot of headaches debugging in the past)

[excuse my mockup]

I’m unsure of this. Does a new entry just appear at the bottom of the modifier / group when you unhide it? I think for a complex model this could be impractical on the output node as well if you have 20+ local attributes floating around. I’d probably just use the Set Attribute node to specifiy the data and attribute name. It keeps it much more visual and connection based which feels fundamentally important to visual programming. If you want to control the name from outside on the modifier then you can just connect from a group input to the Set Attribute name.

Why?
A float list has length N, and a single float has length 1. To an artist this difference is pretty arbitrary. It’s quite likely that a large system is made on a single element to begin with and then single inputs are just exchanged for the attributes to create the full array. If sockets are differentiated does this mean that they would need different nodes to be processed?

Another design challenge is how list length is interpolated when lengths don’t match. Normally I’d think a shorter list would repeat the final element until it reaches the longer list length but it would be good to have an explicit node that could match attribute lengths according to some options if needed for specific use cases (Long (repeats shorter final elements), short (culls lists to match length of shortest), cycle (repeats entire shorter list until long list length is reached)).
In my mockup above the bezier segment node would need to be matching the initial two lists of 25 elements to the final two sockets containing 1 vector. The expected behaviour there is just to repeat the shorter list so each bezier receives an element for each point without missing out any of the input data.

Great proposal. It definitely feels more grounded than the first Attribute Sockets proposal in a good way!

4 Likes

Immediate impression: expandable sockets is a UI shortcut for “GetAttribute” nodes, which would be a separate node with a Geometry + String input (or attribute selector for built-ins/modifier context)
(Good practice to explain the feature in primitive functions, even when the UI tricks are useful)

Advantage of the design (this should be stated clearly):
It’s easy to trace the origin of an attribute to the geometry because they are both displayed as “outputs” of a single node (as opposed to GetAttribute where the geometry is on the input side).

This is “only” a visual aid: If there is an underlying logical connection in the data then it could just as well be inferred from a GetAttribute node. An attribute output socket would declare a geometry socket as its origin.

  • Are attribute connections allowed to “skip” a node? E.g.

    The attribute has only been interpolated, so if “Mushrooms” is only a reference (name) then it still exists on the Subdivide node output geometry.
    Needs some really good visual communication to explain when attributes are connected from the wrong geometry.

    • What happens if you try to connect attributes while geometry is unconnected?
    • Does everything become invalid when you connect a different geometry, or does it try to replace all attribute connections automatically?
    • How does that work across group interfaces: Does the group declare which geometry input an attribute input has to be part of? Could get quite complicated.

    This gif seems to suggest that skipping is not allowed, and it will be corrected automatically.
    geometry-nodes-connections

    However, it’s not clear how that automation would work:

    • Is it just string based?
    • If it knows that an output is an attribute of some geometry, it still needs to map that geometry to the relevant input of the target node. Lots of room for corner cases here …
  • Not sure i understand the Attribute Outputs section.
    May be a naming issue: would only call it an “attribute” if it’s part of geometry, otherwise is just an array. I prefer to keep it simple: Once you accessed an attribute it’s disassociated from the geometry, it beomes just a list of values.

  • Is “Adding attributes to geometry sockets” an optimization feature? I.e. to reduce amount of attributes that are not needded. If so, then i would try to automate by detecting “used attributes” from outputs, similar to what happens in shaders.
    Not so easy when the geometry leaves the tree context after the modifier, so there should still be an explicit “save this attribute” node to expose internal values. But better do that explicitly that through a complicated node output UI feature.

So in summary: I would keep it simple and say that “attribute” outputs are lists as well. Any association with a particular geometry should be optional “metadata”: You can pipe in any list socket to an input, but the system may look at the metadata where it is available and give hints and warnings if data is likely to be a mismatch. There would be a “semantics” layer on top of the raw data processing:

  • Nodes declare which inputs should be attributes of a geometry input and in which domain.
  • Nodes declare what geometry and domain an output is part of.
  • Nodes declare when a geometry output inherits attributes from an input.

All of these would be optional metadata to enable advanced UI tools and error reporting, but not necessary for raw data processing. The node system itself only needs to check for basic validity like matching data types and array length, but the rest is up to the user.

6 Likes

Hi @Erindale ,

Is pop short for populate here?

No, Pop means get + remove from geometry.

Are local just attributes set inside the node tree itself?

Yes

What’s the benefit of separating Built-In, Input and Local?

They are different. So users should tell them apart. Whether it is by showing them as a category or different colors, or something else. We need the same distinction in the spreadsheet editor.

Little recap:

  • Built-in Attributes: always available (position, material_id, …).
  • Input Attributes: mapped from the modifier to the active object based.
  • Local Attributes: created inside the nodetree, are removed outside the nodetree unless explicitly added to the geometry output. In this case they are mapped outside to an attribute from the active object.

Note that object/collection info violates this a bit and need some exceptions to be handled well.

Is “Geometry Expand” necessary?

Not necessary per-se. But it can be convenient to connect a geometry that comes from a Reroute for instance.

Really cool idea [adding local attributes to a geometry socket]. Some more clarity to the design [is needed].

Agree. This is part of the UI/UX pass that this proposal can still get. You could use e.g., a thick green border for the added ones, so you know they are not part of the geometry (besides the +).

Numbers of element in a list.

This should be visible in the socket inspector.

Whether we also need this in the node/socket I don’t know. I think it is more important to indicate that there is a size mismatch, not the number of elements necessarily.

Does a new entry just appear at the bottom of the modifier / group when you unhide it?

Yes.

Why to treat lists as any different than scalars?

I agree that for the output they are indistinguishable. We do need a way to differentiate them on the input. For example, the Point Distribute node has two float inputs: Density Max and Density. Density Max only supports a single element, while Density can have different elements per domain.

If we use the domain icon near the sockets that may be enough to tell a scalar apart from a list.

2 Likes

Hi @LukasTonne

Let me start by reiterate that what is outputted from the attribute socket is a list. Not a reference.

Are attribute connections allowed to “skip” a node?

Technically yes, but that means they will not work as expected if the topology changed in between. In “your” example the multiply would have less elements than the subdivided one, leading to mismatch of the ids and impredictable results.

Once you accessed an attribute it’s disassociated from the geometry, it beomes just a list of values.

+1

‘Is “Adding attributes to geometry sockets” an optimization feature’

It is more about having users being in control of the data that is created, as oppose to have data (attributes / spreadsheet columns) created automatically without an easy way to track where do they come from / when they were created.

3 Likes

So it’s a cut rather than a copy operation.

2 Likes

For Blender 5.0 i’d like you to imagine a node editor where material output, geometry output, simulation nodes etc live happily next to each other (EVERYTHING nodes). Let the noodle decide if a tree makes sense or not.

5 Likes

Hi there, I have a proposal which partially overlaps with this topic on RCS which may be relevant to the discussion, perhaps some of the ideas I suggest could be relevant to this proposal:

Prototype!

Hey, this week I worked on a prototype for this proposal. Builds are available on the buildbot (they are still compiling at the time of writing this). The code is in the temp-geometry-nodes-expandable-geometry-socket-prototype branch.

The prototype is still quite rough, but it already contains some of the most important features necessary to test the workflow. I had to make some compromises, to be able to get this far within one week. Below I show what is possible in todays prototype. I few more things might be added over the weekend or next week.


Simple deformation of a mesh with a noise texture. The Geometry Expander node is used to get a list of all the vertex positions.

Simple point distribution and instancing. The Rotation output contains a list that contains the euler rotation for each point.

It’s possible to output a list to the modifier. The modifier then specifies which attribute to write the list to. Currently, it can only write the attributes to mesh vertices. As in all other cases, if lists lengths don’t match, the list shortened or repeated. There are no node warnings when lengths don’t match yet.

A second extrude node can just use the selection created by the first extrude node.

If one wants to use a third extrude node, that uses a selection created by the first extrude node, one has to do the following: Click on the plus in the output socket that is supposed to be used. Insert a Geometry Expander before the third extrude node. Add the selection output to the Geometry Expander.
Connecting selection from the first node to the third does not produce useful results, because the face indices don’t match anymore (because the second extrude node created new faces).

It’s possible to specify an attribute name in the modifier ui that can then be accessed by the Geometry Expander. For that one has to create a String input in the Group Input node. This is a bit of a workaround for now, because it should be possible to get the list from the attribute directly in the Group Input node. I’ll still try to add that if time permits.


Some more info:

  • Node groups are generally not supported in this prototype. The proposal requires some fairly fundamental changes to the way node groups are evaluated, that I did not have enough time for to implement.
  • “Function nodes” (e.g. math nodes etc) should generally work. The Switch node does not work on lists yet.
  • Geometry nodes not shown in the screenshots, generally don’t work with lists yet.
21 Likes

Good to have prototypes of both approaches now to compare them fairly.
I did a comparison of both on the case of a delaunay triangulation node over on the fields topic.
Just though I would link it here too.

3 Likes

Still not able to find a Windows build for this proposal, is something wrong with the buildbot?

EDIT: The build is up now!


Practical use:
Procedural phyllotaxis using the expandable sockets patch.
I have to admit, at first I had my doubts, as I actually liked the text fields for attributes. However, this new system is so much easier for people who did not create your tree to follow and understand. The old method was very messy:

9 Likes