This document proposes changes to how instances are handled in geometry nodes. First, it explains the current state and why it works this way. Then it argues why the original reasoning is flawed nowadays. Lastly, there is a new proposal for how we could move forward.
Current State
Currently, the main rule for how instances in geometry nodes are handled is that they are always automatically made real when necessary. For example, the Set Position node always makes instances real before updating the position.
There were a few reasons for why we decided it should behave this way:
- Instancing is used only for performance reasons and should be an implementation detail.
- We thought about having Collection Nodes. Those would probably work on the instance/object/collection level, whereas geometry nodes only works with the geometry data directly.
Issues with the Current Workflow
I think the reasons mentioned above are flawed nowadays or at least out of touch with reality. Below are arguments that speak against our original reasoning.
Performance and Implementation Detail
It is true that one of the main reasons for using instances is that they are typically more time and space efficient. In fact, that is so true that artists working on complex scenes often have to make conscious decisions about what is instanced and what is not. Users have to make the trade-off between the ability to e.g. deform every piece of geometry individually which may result in more visually pleasing results, and limits of hardware and time. In more complex scenes this trade off should not be done automatically but by the artist. Therefore, instancing should not be treated as an implementation detail.
Currently, instancing is not really an implementation detail anyway, since it is possible to see when something is an instance and when not in the spreadsheet. If instances were truly an implementation detail, then the instances category in the spreadsheet shouldn’t exist, and it should always only show the realized geometry data.
However, even if instances were a pure implementation detail, completely abstracted away from the user, it would be a very leaky abstraction, because users (1) still want to know when something is an instance for the reasons mentioned above and (2) it becomes very apparent when something suddenly becomes real, because processing time explodes.
Also, just because we expose the concept of instances to the user, it does not mean that we can’t do more internal instancing (or data sharing) that the user is not necessarily aware of. In fact, we are doing that already and I hope we can do more of that in the future (e.g. sharing attribute data between geometries with a copy-on-write system similar to what we do for geometry components already).
Other Uses of Instancing
Next, I’d like to argue that performance is not the only use case for instancing, especially now that we support geometry instancing (instead of just object and collection instancing).
Having instances allows embedding multiple geometries in a single geometry, while keeping their individual geometry data separate. That gives us a list of geometries that can be used by nodes like the newly proposed Instance on Points node, that supports putting different instances on every point.
Furthermore, being able to access the instances separately allows artists to instance first and then move the instances around afterwards. This may be more user friendly compared to forcing the user to first position points correctly and only instance in the end. Instancing first would also make supporting nodes easy that push instances apart so that they don’t overlap. That’s especially true when different instances have different sizes.
Another thing we don’t support yet but should eventually are attributes on instances. Again it would probably be easier for artists to instance first, and then add attributes to the instances. Those attributes could then be used either by later processing steps or the renderers. Often it is conceptually important whether an attribute is on the instance or on e.g. every vertex. Shaders are build with that in mind.
Rigid body simulations may also benefit from the ability to have multiple instances in a single geometry. A “rigid body solver” node could take in a geometry, simulate all the instances as individual rigid body objects and output a new geometry in which the instances have been moved.
With geometry instancing we got the ability to create nested instances. For example, a forest geometry contains a bunch of tree instances, whereas each tree contains many leave instances. All of that could be built in a single geometry node network. For rendering it does not matter if the list of instances is flattened in the end. However, we could support exporting an entire nested structure for file types that support it. For that we probably want a way to set the name of an instance. The other direction is possible as well. We could import an entire hierarchy into a geometry with nested instances.
These nested instances are also what differentiates the instances component from a simple geometry list socket type that we may want to have at some point.
Yet another use case of instancing besides improving performance is to use geometry nodes for the initial scattering of objects. Then one can use the existing Make Instances Real operator to create separate objects for the individual instances. Their position can then be further fine tuned manually by the artist. Nested instances could even be created with the correct parent relationships.
Collection Nodes
Not much time has been spent figuring out how collection nodes would work exactly. However, as mentioned in a blog post from June, we decided that simulation solvers will be integrated in the pipeline at the geometry nodes level. So at least for that, collection nodes would not be necessary anymore. There may be other reasons to get collection nodes in the future, but for now I wouldn’t bet on it now.
Proposal
The proposal is to treat the instances component as a first-class citizen alongside the other component types: mesh, curve, point cloud, volume and the future hair type. Similar to the point cloud component, the instances component has a single attribute domain: Point. The difference to a point cloud is that every element in the instances component references some other geometry and has a full transformation matrix built-in.
Nodes should generally avoid making instances real implicitly. Instead there will be a separate Realize Instances node that has a geometry input and output that can be used in the general case. Besides that there are a few rules for handling of instances in nodes:
- Nodes that modify geometry should ignore instances by default (e.g. Extrude).
- Some nodes may get an option to process each instance separately (e.g. Edge Split, Subdivision Surface). In those cases, the output geometry will still contain instances. Each geometry referenced by the instances is processed separately, independent of the instance attributes (such as position).
- Nodes that produce completely new geometry based on some input geometry (e.g. Point Distribute, Boolean) should not ignore instances. Instead, one should generally try to make the node work more efficiently by e.g. processing the individual instances in parallel. For those nodes it would usually be less efficient to realize instances before further processing.
- Nodes that modify attributes should not make instances real, but apply the changes to the instance attributes (e.g. the Set Position node sets the position of instances instead of the position of every vertex).
One open question is whether instances should automatically become real when there are other modifiers after the geometry nodes modifier. Currently, I’d argue with no, because if one wanted to make instances real, one could just add a Realize Instances node.
Implications
The main downside of this proposal is that the user now has to be aware of when something is an instance and may have to insert a Realize Instances node in places where it was not necessary before.
Luckily, there are only a few nodes that actually create new instances (currently Point Instance, Object Info and Collection Info). So the number of places where one had to insert a Realize Instances node is quite small. It is so small, that we could even consider adding an option to output realized geometry from those nodes directly.
We already of multiple ways to show the user when something is an instance: spreadsheet and socket inspection. If some nodes will ignore instances in the future, those may also show a warning mentioning that an explicit Realize Instances node is necessary.
From my perspective, this small disadvantage is well worth it when compared to the possible workflow improvements we get by embracing instances as a first-class citizen in geometry nodes.
