2024-05-13 Geometry Nodes Workshop Notes

Meetings
1

This workshop took place at the Blender head quarters in Amsterdam.

Present

  • Bastien Montagne (assets)
  • Brecht Van Lommel (assets)
  • Dalai Felinto (~2 days)
  • Erindale Woodford (~4 days)
  • Falk David (~2 days)
  • Jacques Lucke
  • Lukas Tönne (~3 days)
  • Nathan Vegdahl (private node groups)
  • Sergey Sharybin (assets)
  • Simon Thommes (~3 days)
  • Pablo Vazquez (gizmos)

Notes

These are the meeting notes we took during the workshop. There is also a blog post summarizing the different topics.

Gizmos

  • descriptions/tooltip on hover in 3D view
  • snapping on ctrl
  • Transform Gizmo
    • IN: Value, Base Transform, OUT: Transform
    • supported backward-propagation:
      • Required: Combine Transform
      • eventually do smarter back-propagation based on link path
  • Flow Visualization
    • generally get rid of icon and pinning
    • keep icon/pinning only on the node where the actual value is, which is being controlled
    • visualize links
      • explore double noodle
    • draw gizmo links for as far as compatible with back-propagation
  • Gizmo Panel
    • panel in node-group to manage/group the used gizmos and attach to context so the user can toggle them based on that, rather than what inputs they control
      • nice-to-have, since it isn’t required for the first usable version
    • panel in group-node/modifier to toggle gizmo visibility on user side
      • required for general visibility control
  • Interaction Modes/Versions
    • let user choose from different interaction modes depending on use-case
    • does not affect node-tree (sockets)
    • support for gizmo groups?
      • e.g. transform/scale cage
    • control gizmo ‘group’ visibility (e.g. array with first/second or first/last element - rotation+offset)
    • user interaction
      • only expose hard-coded options for individual builtin gizmos in the viewport options
      • toggle between different gizmos/gizmo groups that are defined in the node-tree
  • Complex Backwards Propagation
    • eventually do smarter back-propagation based on link path

Baking

  • Bake to .blend
    • Should bake to .blend by default, instead of writing to external cache
      • Need a new toggle next to the file path that controls whether the bake should be written to disk
      • That toggle ideally exists on all the levels where we have the path (per bake/modifier/project)
    • After first version: Operators to write packed bake data to disk or pack from disk.
      • Packing baked data should be part of the existing Pack operator
    • If the bake is packed, we need a solution for overriding.
      • Would need library overrides for packed bake data.
      • Or force the use of external bakes for overrides.
  • High Level Overview
    • In outliner.
      • Was discussed in UI module meeting that it’s ok to have a new view mode for bakes.
    • It generally doesn’t make sense to rebake linked data.
      • There can be a view option to show bakes from linked objects (greyed out).
    • For details see previous workshop notes.
  • For simulation playback something like the preview frame range would be useful
    • It should automatically use the earliest simulation start in the scene
    • Different options:
      • Change the preview frame range icon into a dropdown with Scene Time, Preview Range, Simulation Range items
        • The simulation range would be automatically derived from all simulations in the scene
      • Have a button or context menu entry to set the preview range to the simulation frame range derived from all simulations
        • Potentially only update the start frame
    • Need to be able to switch between a hierarchy and flat view
      • More generically we could support disabling collections/objects/modifiers/node groups from the hierarchy (somewhat similar to existing drawing modes in viewlayer display mode)
  • Tagging
    • Want to be able tag bakes
      • Allows e.g. rebaking everything that has a specific tag at once.
    • Tags could be stored on the nodes directly and/or per node instance (on the modifier)
      • In practice we probably need both.
      • Asset authors could give some initial tags that are then further customized by the user.
    • Tags on node and node instances could be concatenated to a final list of tags, or stay separate.
      • Union them alltogether seems good enough for now.
      • Might become necessary to filter specifically on node or node instance tags.
    • Need two lists of tags in the sidebar when a bake node is selected (or simulation zone).
    • Tagging in the outliner
      • Definitely want to see all tags.
      • Can add and remove tags only on the object level.
  • Sometimes it’s nice if cache invalidation directly causes a simulation reset
    • Currently, the simulation keeps playing in an invalid state.
      • Often that’s useful, e.g. when emitting particles from an object and moving that object around in the 3d view.
    • Together with the simulation frame range, one could use shift+left-arrow to jump to the first simulation frame which also resets the cache if it’s invalid.
    • Not totally clear yet if that really solves the problem, or if an option to automatically reset the cache when it’s invalidated is necessary.
  • After duplicating baked object, it’s not possible to change the bake path on the duplicate without freeing the bake for the original object.
    • The problem is that the bake path is read-only when there is baked data.
    • An initial solution could be to grey it out, but to keep it changeable.

Path Variables

  • Not string drivers (those are animation evaluated by the depsgraph)
  • Project level, user pref level, context level (filename, modifier_path, bake_id etc.)
  • user-defined variables at some point later
  • support access from python API
    • pattern
    • previously evaluated path
    • current evaluation of pattern
    • on-the-fly evaluation of a pettern in specific context
  • bake path:
    • resolve bake path on each bake and lock in baked state
    • requires storing pattern and evaluated path separately (two path text fields)
    • actual resolved path read-only
    • needs to be clearable without removing cache files (e.g. on object duplication)
  • potential filepaths that should support this
    • modifier bake path
    • node bake path
    • import node filepath
    • render output
    • image filepath (reading)
  • Example setup for baking
        * SCENE:      //blendcache_${FILE_NAME}
    * MODIFIER:   ${SCENE_CACHE}/${OBJECT_NAME}-${MODIFIER_NAME}
    * NODE:       ${MODIFIER_CACHE}/$BAKE_ID
  • show error when path resolution does not work
    • with useful tooltip
  • investigate displaying resolved path and editing pattern (like driver expressions)

Rename Sockets in Nodes

  • Problem: It’s inconvenient to have to go to the side bar to change the name of various sockets.
    • It’s especially a problem because the name is created in the node UI, but can’t be changed there.
  • Limit discussion to just renaming for now.
    • In theory one could also support removing and moving sockets without the sidebar.
  • Should only apply to “custom” sockets that can be renamed already anyway, not just any socket.
    • That includes sockets on group input/output, simulation/repeat zone, bake node, menu switch.
  • Renaming shouldn’t happen on a single click, because that would make it annoying to move the nodes.
    • Instead use double-click and/or ctrl+click
      • Double-click is tricky to do currently in the UI code
    • Additionally, can also support right-click > rename.
  • There should be a highlight indicating which sockets can be renamed.
    • Could potentially be a box around the socket name, similar to what you’d get in a list view when hovering over the items.
    • Same thing could apply to breadcrumbs for going up to some node group (or even renaming them).
  • A problem is that some renameable sockets have a value (e.g. unlinked float input in bake node).
    • So they don’t show the label in a way that one can click on to rename.
    • Right-click > rename can still work.
    • This case should not happen often in practice.
  • Renaming node groups sockets from group nodes should not be supported
    • Breaks the encapsulation of node groups.
    • Could be allowed for “single-use node groups”.

Tools for Node Tree UX

  • Node Tree Comments
    • Not possible to have multi line inputs currently!?
    • Check if Julian has a PR for multi-line inputs.
      • The previous work was mostly on multi line labels, not inputs
      • Need to investigate adding a new uiBut for a text area
  • Custom zone colors
    • Similar to how one can change node colors
    • Input and output can share the same color
  • Modal operator to make space in the node tree
    • Everything to the left/right of the mouse is moved by the distance the mouse moves
    • As an alternative to the Auto Offset feature which is less predictable
    • “Slide” tool
    • Clicking on node could use upstream/downstream instead of placement (test?)
    • Let UI module decide on a shortcut
  • Link Portals
    • Links can become portals.
    • Would need to show some stub at either side
      image
    • Overlay settings should have a global override to show all the portals as normal links.
    • When a node is selected that has portals, those link should be shown (maybe in some dimmed state).
    • Low priority
  • Link drag-search
    • Insert onto existing noodles with modifier key
    • Connect to existing sockets with link drag search.
      • Only applies when dragging from an input socket.
      • Link-drag-search should include labels (if there are any) of input nodes and reroutes.
      • We were considering if this should create a portal by default, but probably not.
  • Frames
    • Ctrl+J should parent the new frame to the common root frame of all selected nodes if there is any
  • Auto-offset
    • Change default offset from 80px to 20px (later we agreed on 40 instead)
  • Improved Search
    • by type
    • by label
    • by node group usages
    • by node group input uses
  • Drag&Drop attributes from attributes panel
    • Creates group input
  • Operator to swap input value node and group input node
  • Ignore link-insert when the connection is invalid (red link)
    • Highlight red links before insert
  • Add Separators to node(-group) UI
    • Sample node could use this to clarify input evaluation context better
  • Add empty node group node (+new button)
  • Viewer Node
    • When ctrl-shift-clicking to add a viewer, and a viewer exists already, it should be moved right next to where the user clicked
    • Re-using viewer nodes interferes with the goal of allowing multiple viewers.
      • Idea: “Pin” viewer nodes to prevent them from getting repurposed.
  • Assets in Menu:
    • Installing node group asset libraries increases the size of the Add menu a lot.
    • User should be able to selectively discard node assets/catalogs from the add menu on user end
      • Asset authors shouldn’t be able to enforce that the assets show up in the menu
  • Node Wrangler:
    • Parts of node wrangler should be merged into main (also see #121749)
    • Lazy connect/Make Link/Mix:
      • Might not need the different colors
      • needs to be checked with the UI module if we can justify adding more colors

Asset Embedding

  • A proposal written before the workshop was discussed.
    • Generally agreed upon the direction.
    • Proposal needs to be adapted a bit to avoid changing the definition of what a library is too much.
      • Solving this involves some trickery with ID names to avoid name collisions, but can work.
  • Having a library per embedded data-block is a conceptual problem, because libraries are supposed to be file references.
    • Data-blocks coming from the same .blend file should share the same library
    • If two data-blocks from the same .blend file with different deep hashes are embedded, their ID name should be deduplicated
      • The original name still has to be stored, can be used for versioning etc.
    • Rename has to happen whenever loading data-blocks, even data-blocks coming from linked files.
    • Two data-blocks with the same deep hash, can have different ID.name, but generally their original should be used.
    • There is a conflict when data-blocks with the same deep hash exist in multiple files.
      • They should still be deduplicated, but the library path is ambigous.
    • Multiple libraries with the same filepath are not supported currently.

Menu Socket

  • The current implementation is not fully implementing the design yet.
    • This was known from the start but we decided to do it anyway, to get Menu sockets in earlier.
    • It requires dynamic socket types.
  • We should find a way to inform the user that node groups can be used to use the menu multiple times.
    • For example, when having this invalid menu socket connection, there should be an info/error message on the menu switch nodes.
      image
      image875×400 32.9 KB
    • Showing error messages for invalid links should be a more general thing.
      • The error should generally be shown on the target node (instead of the origin node or link)
      • That’s probably the most useful place, because the user is already looking at this node in most cases
        image
  • If implementing both of these things is not good enough, we can consider adding a new data-block type for menus
    • This data-block type would be referenced by the Menu Switch nodes and serves as ground truth.
    • Implementing a new data-block type is a fairly big task and requires approval by more people outside of geometry nodes.
  • We were checking the design for dynamic socket visibility again, and it still makes sense.

Field Context

  • Rename Attribute to Target Attribute in Sample Nodes
    • Add info to tooltip on hover
  • Show domain a field is evaluated on in tooltip on input sockets.
    image
  • Use socket selection to highlight evaluation context
    • Highlight downstream (only) flow of fields up to where they are evaluated
      • upstream highlighting can add confusion about which inputs influence the field and remove nuance (?)
      • test highlighting up-/down stream based on in-/output socket or highlighting both always
      • highlight should retain noodle color
      • could use noodle outline for highlighting (thicker than standard)
    • Highlight evaluation geometry context
    • display additional (e.g. domain) info on hover (including encapsulated evaluation in node-groups)
    • This could be generalized more to allow for different kinds of flow visualizations
      • can be added as node tree overlay settings

Lists

  • Definition: “Array of values that is not tied to a geometry or domain”
  • Treated similar to grids in evaluation
  • Fields are trivially convertible to lists (broadcasting rules)
  • Multifunctions can combine fields and lists based on simple index
  • List Fields:
    • E.g. a list for every point
    • TBD: can be stored as attribute?
  • Nested Lists:
    • Could be supported in theory but complicates type system and there are performance concerns
    • Flat lists are generally way more efficient
    • Nested lists could be worked around using one plain list and an offsets list.
  • Conversion list to field:
    • TBD: require a conversion node for lists → fields?
      • If implicit conversion is allowed: show warning if list length != domain size.

Socket Shapes

  • Current set of socket symbols (diamond, circle, diamond with dot) does not work any more with new grid and list types.
  • Communicating current state together with potential state gets confusing.
  • Suggestion to remove the diamond+dot symbol (“is single value but can be field”), to simplify visual language
  • Asterisk to show undetermined field type (only possible when connected to group input/output)
  • Socket shapes indicate concrete field type (single value, field, grid, list)
  • Dots to show changeable field type (local node behavior, not propagated)
    (this is inverse of current behavior: dot is the current single-value type, diamond is the potential type)
  • Suggestion: use circle for any non-field types (“data”: grids, lists, …)
  • Alternative: Use circles for all “data” sockets (single values, grids, lists, strings)
  • How to show possible connections without relying on shapes?
    • Tooltips, text next to socket, icons next to socket
    • Maybe additional overlay in 3D view
    • Requires improved error handling beyond what is indicated by socket shapes
  • Alternative: Use 3 socket shapes: existing circle (single data), diamond (field), square (detached containers of data: lists, grids, etc.)
  • Use Asterisk only in special cases for learning that sockets can be of different type when nothing is connected
  • Overall, this discussion turned out to be more difficult than expected.
    • We did not come to a conclusion.
    • Could not come up with a full proposal yet.
    • Especially tricky because sometimes sockets support subsets of all available types which are hard to represent with sockets.
    • Next task for Jacques: Investigate ways to not rely on socket shapes too much while still showing the relevant information when it’s necessary.

Single Use/Private Node-Groups

  • Use-cases/Goals
    • Presenting Node-group data-blocks in add menu depending on context
    • Menu switch encapsulation (private)
    • Node-tree organisation regardless of datablock management (single use)
  • private node-groups can solve the same use-cases as single use ones
    • Couldn’t really come up with a use case that definitely required single-use node groups which would always be deep-copied.
  • new flag in usage panel ‘Public/Menu/Group/Local/Node’
    • only affects whether or not it is shown in add menus (including search and link-drag-search)
    • still show in id pickers
    • default:
      • local for ctrl+g
      • non-local for new modifier/tool (versioning for existing)
  • operator to create a deep copy of a node-group node (= duplicate + make single user)
  • Alternative Solution
    • Global implementation of ‘namespace hierarchies’ to allow more hierarchichal integration of data-blocks in general
      • Recognized that the problem of “private data-blocks” is not unique to node groups.
      • It can also make sense for a node group to “own” an object for example, which shouldn’t be exposed elsewhere.

Grease Pencil

  • Join Geometry: In the spreadsheet we already show the layers in their index order (not layer stack order, i.e. bottom-top)
    • We should also join the geometries in this index order (in the order you plug them into the node)
  • Realize Instances: Also use the index order to concatenate the layers into one grease pencil geometry
  • To get the curves geometry out of grease pencil there would be a new node: “Grease Pencil to Curves”
    • This node gets a grease pencil geometry and a layer selection
    • It outputs curves instances (ideally named with the corresponding layer name) (socket name “Curve Instances”, just like “string to curves”)
  • Layer settings are not stored as attributes and will get lost for now
    • We could output some of them as anonymous attributes on e.g. the Grease Pencil to Curves Instances node
  • To combine Curves to Grease Pencil there could be a “Curves to Grease Pencil” node
    • Named instances are used to create layers and append them to a new Grease Pencil.
    • Could have a selection input
    • Needs to deduplicate names
  • Grease Pencil only works with poly curves atm!
    • Should be solved by the time GPv3 lands
  • layer properties could be converted to attributes before evaluation
    • Also means that we have to make sure to use attributes for rendering
  • Merging layers: Separate node (“Merge Layers”)
    • Use a Group ID to define the layers that will be merged togerther
    • Maybe this could also be generlized to a “Merge Instances” node
  • Looping over layers and doing stuff to curves:
    • Idea 1 would be to have a “For each geometry instance” zone
      • seperate greasepencil to curve instances before and combine after
    • Idea 2: Have a special “For each layer” zone
  • Because we only look at the current frame in geometry nodes, there are cases where we loose information on other frames
    • This can break onion skinning
    • One idea: look at original geometry

Custom Viewer

  • Goals
    • Want to show potentially multiple columns in the spreadsheet
      • Allows comparing values
    • Want to have extra preprocessing on the geometry before it’s shown in the viewport
      • Density grids can be shown as boxes, velocity grids as arrows, etc.
    • Some way of marking ‘static’ viewers besides the ‘dynamic’ one used for ctrl+shift+click
  • The shortcut for adding/connecting a viewer doesn’t work well anymore, because there would be multiple kinds of viewers.
    • Always use last active ‘dynamic’ viewer (including custom ones)
    • ~~Pie Menus could be useful in practive
      • it’s not clear how it is populated and what happens when it is full
      • Setting it up in the preferences is tricky if the set if viewers differs depending on the opened .blend file.
      • Could enforce that custom viewers have to be part of the asset library to be able to put them into the pie menu.
        • Then they are always available
        • Would only really work if asset pushing was easy~~
    • Pie menu construction and key binding should be a general feature, not specific to viewer nodes.
  • We’ll also need a new kind of viewer to add data directly in the node editor
    • More important once we get lists
  • Need to be able to tag a node group as being a viewer
    • Options:
      • Flag on the node group
      • Could use a heuristic that checks that there are no output sockets
        • Does not work so well when we have node groups that just bake/export data
      • Have a different kind of group that’s specialized to creating a viewer
  • Viewer node groups should show up in the Add > Output menu, next the the existing Viewer node
  • Workflow to create a custom viewer:
    1. Select a viewer node.
    2. Hit ctrl+G to create a group
      • That creates a new group that has no output socket and has the viewer flag set already
      • There is a special case when grouping a single node already: it tries to make the group look like the node inside
        • Although this should also be extended to set implicit inputs and color tag
  • Alternative design:
    • Instead of having viewer nodes inside of the viewer node group, one could use group outputs.
      • No obvious benefit besides potentially less boilerplate code because outputs are a list already
  • If multiple viewers can be active inside a viewer group, the same should work in regular node trees.
    • Any active viewer shows up in viewport and spreadsheet.
    • Viewers can be muted to disable them (currently muting viewers is not possible).
    • Problem: Spreadsheet only shows one geometry at a time, multiple viewers can conflict.

For-Each Zones

  • For-Each Geometry Element Zone
    • Input Node
      • domain dropdown
      • Inputs:
        • Geometry
        • Selection
        • Group ID: once lists are supported to internally pass the list of evaluated attributes for the group elements
        • Arbitrary number of field inputs evaluated on the chosen domain
      • Outputs:
        • Element Geometry: Disconnected loop element of the input geoemtry
        • Loop Index/ Group ID
        • Single Values of evaluated attributes/list for groups
    • Output Node
      • Geometry Output: identical to zone input geometry + anonymous attributes that are passed out
      • In/Out:
        • Arbitrary number of value inputs that are stored and passed as anonymous attribute fields
        • Arbitrary number of geometries to be joined together in additional output
        • Arbitrary amount of fields per geometry socket
          • Problem: Matching which field should be evalua
            • probably do the same logic as for simulation zone, where all fields only correspond to the last previous geometry socket
    • pass in multiple geometries to process elements in parallel (e.g. matching Indices/group IDs)
      • iterate over union of elements (not just elements of first geometry)
      • these would all need the same kind of functionality and corresponding sockets on in/output nodes as the first geometry input
  • Other types for loops (Basic parallel iteration, list elements, (grid voxels)

Bugs

  • Choosing an existing geometry nodes data block on an object without geo nodes modifier does nothing (#84927).
22 Likes
3

Is it planned to expose enums in built-in nodes as menu sockets? That’s the thing I’m waiting for. I have couple of modifiers that would benefit from being able to choose domain type

1 Like
4

About long Add menus, I agree that for me it became unusable, and I think it’s in the state now that nobody actually uses it, you’re just starting to type and Add menu basically acts as a search for node operator. For new users they’re intimidating as well. I think the biggest problem is the number of low-level nodes exposed there. I know I’m swimming against the current with this idea, given that the module has a design for splitting nodes, but I’m all for combining them.

I see that search by type was discussed, and it’s the most vital UX thing missing in node editors imho. Whole problem started with the inability to search for “Multiply” and “Dot Product” and etc. So, if we were able to search for them, if we searched for “Add” and result showed Math >> Add, would it still be necessary to split the operators into different nodes?

I don’t think there are benefits of splitting (on user-level), but on UX side it makes a lotta sense when they’re in one node. Enum is going to be better for switching type than any operator, and it will slim down add menu by A LOT.

Like for example if all new Matrix nodes were combined into Matrix Math node with different modes, like Vector Math, and you could still search for them individually when you start typing in Add menu, and can sit in the same menu as other math nodes. Besides not taking up space in add menu it will make them more “approachable” for new users, and easier to swap between them.

But some more can be combined as well. For example all Corners of Edge, Edges of Corner nodes can become one Mesh Topology node and benefit from all the same things, as well as just straight up removing one whole submenu in Mesh menu.

  • Float, RGB, and Vector Curves can be modes of one node
  • All Rotation nodes
  • Rotate, Scale, Translate Instances
  • I would even make one Convert Geometry node in which all “Mesh to Curve”, “Curve to Points” nodes are modes.
3 Likes
5

Great topics!
Is there a rough timeline for more particle stuff like force fields, solvers?

4 Likes
6

Do you think it can be considered being able to select and drag noodles in any point and not just their ends?
It would speed up a lot the process of re-connecting and detaching them. I made a proposal some time ago where Jeroen Bakker seems interested into: Right-Click Select — Blender Community

1 Like
7

There is this old patch by lone_noel that seemed, at the time, pretty mature. Specifically the part that creates a draggable zone for reroutes, so node trees can be fully controlled with the mouse. On the off-chance it doesn’t require too much rework, could it be bundled with the rest of these enhancements ? it looks like pretty low-hanging fruit so it seemed a good time to bring it up again.

https://archive.blender.org/developer/D11209

5 Likes
8

Great! As nodes and dependencies between objects grow, for very large and complex projects, I believe that in the near future it may be necessary to resurrect OOPS at this point.

IMG_0800
IMG_0800602×539 17.8 KB

Thank you so much for the great work you do, I really appreciate you spending a lot of attention on low level features and functions.

7 Likes
9

A multi-line text input control would be very useful for VSE too. The Text strips today do not have any sensible way of making a multi-line text – the code can handle it, and you can assign a multi-line text to text strips from Python (embedding newlines into your text string), but you can’t do that via UI. I was looking around with “surely blender somewhere has a multi-line text area widget”, and was similarly surprised about lack of it.

5 Likes
10

The simulation is somewhat slow in parameters of large size!

10 Likes