Serial Loops [Proposal]

[jacqueslucke]

Serial loops allow the repeated evaluation of nodes. This is necessary for many effects but became significantly more important with simulations recently. That’s because many kinds of simulations do repeated solver steps to reduce e.g. collisions. Currently, people just copy the same nodes many times, but that’s annoying and easily doesn’t allow an arbitrary number of iterations.

Example from the simulation demo file:

Note, the goal is not to provide a solution to e.g. iterating over every point one by one. While it’s technically possible, serial loops are a very inefficient way to do this. That’s what lists and parallel loops will be for in the future.

There are a few features we expect from a serial loop:

• Should be possible to mimic a for, while and do while loop from programming.
• Should support an arbitrary number of values that are looped.

Serial Loop Zones

Similar to simulation zones, we can add a new zone type for serial loops. This works well, because the same restrictions with respect to adding links apply to simulations and serial loops (and parallel loops as well for that matter). I.e. it’s possible to link from the outside into a zone but not the other way around.

The Max Iterations input is a hard limit to how many iterations the loop will do. This makes the loop behave like a simple for loop by default with a fixed number of iterations. Using the Break output one can stop the loop before the maximum number of iterations is reached. Based on the Output Previous socket, the loop will either behave like a while or do while loop. Suggestions for better naming are welcome.

The Max Iterations is still a hard limit even when the Break socket is used. This makes it impossible to accidentally create an infinite loop which is very easy to do otherwise.

The mockups don’t show it, but the Serial Loop Input and Output nodes should have empty in and outputs that allow adding more sockets, just like the simulation nodes.

[LazyDodo]

What would you say is the primary use of Max Iterations is? Is it to facilitate easy for(int x=0; x<10; x++) style loops, or is it a safety net for when you made a mistake and the only thing preventing the user from having to kill the blender process and losing some of their work is Max Iterations hitting providing relief from their mistake?

If it’s the for loop, i think it’s great, it’s easy, it’s user friendly, people will grasp it right out of the gate, don’t change it, it’s great the way you propose it. However it’ll get in the way of do { ....} while(...) style loops where the upper bound is unknown, there should probably be an output that signals why a loop was exited so you know if the algorithm ran to completion or was stopped due to max_iterations being reached and you need to maybe increase its value.

However… if it’s to prevent blender lock up, I’m not convinced it’s the right solution, there’s plenty of other places you could do “expensive” stuff in either the RAM or CPU departments are they all going to get their own escape hatches? I’d rather see a generalized option to stop a geonodes graph from evaluating regardless if it’s in a serial loop or not. Though either a stop button or a max executing time for a node graph. I do see this as a separate problem that should likely be solved outside of this proposal. Serial loops make the problem more visible, but you could easily chain a few sub divs without loops and run into the same issue. (Picking on subdiv here, since it’s low hanging fruit, but it’s far from the only trouble maker)

even if we do use it as a safety net, I’m not convinced how well it’s gonna work, some operations are more expensive than others running 10 subdivs on a cube vs running 10 subdivs on a suzanne will already have rather different performance characteristics. Finding the right value for this escape hatch will require some educated guesswork, and you get it wrong, you’re back to killing blender…

[Bobo_The_Imp]

@jacqueslucke

Based on the Output Previous socket, the loop will either behave like a while or do while loop. Suggestions for better naming are welcome.

I think Output previous iteration isn’t bad, but perhaps “Cumulative Output” or “Merge Each Iteration” would be slightly better.

[rbx775]

I dont think its a cumulative output, or a Merged output.
As far as I understand ‘Output Previous’ makes it a while loop when checked, and do while loop when unchecked.
The difference comes from when the break condition is checked, before or after the execution of your loop body.

@jacqueslucke

Serial Loop Zones

Thank you for this clean proposal and your fantastic work so far, loving the progress.

Will we be able todo nested loops?

@jacqueslucke

serial loops are a very inefficient way to do this. That’s what lists and parallel loops will be for

Will parallel loops ‘just’ be a performance upgrade to serial loops?

@jacqueslucke

Output Previous

‘Output Previous’ is already better than ‘while loop’.
What do you think of ‘Pre-evaluation’ or ‘Check pre-execution’ ?

And as you always do, a tooltip on it would be nice: ‘Only in effect when the break condition is met’

[jamez]

This will be very useful! I hope there are still plans to have a way of breaking out of a node tree “cooking”. This will be more important than ever with loops, otherwise Blender will be left hanging a lot of time when you accidentally use too many loop iterations!

I think the plan was to be able to hold escape to cancel out of the current tree operations?

[F_Scociety]

I think for heavy loops it would be cool to have a progress bar and to have it show the current iteration live.
For example when having a erosion loop, you can see how it evolves. And to see how far it is, there could be a progress bar in the bottom.

Though this option should definetly be a toggle and not always on. Another thing to consider, is when doing heavy calculations after the loop. Then i would probably be better to only show the loop output and not the final output for each iteration.

Another option could be, to not auto calculate the loop and maybe expose a button to the node group/ modifier list.
For example when you have multiple heavy loops for a multi step procedural workflow it would be cool to not have it recalculate on changes done in a starting step. But to have you recalculate each step. Oops i think this should go to checkpoints, but still should be considered how checkpoints Cooperate with loops

[jacqueslucke]

@LazyDodo

What would you say is the primary use of Max Iterations is? Is it to facilitate easy for(int x=0; x<10; x++) style loops

Yes, that’s the primary use.

I think it can help prevent accidentally having too many loop iterations in common cases while setting up the loop, but it certainly isn’t a solution for all cases. A solution for that would be independent of loops and is thus out of scope for this proposal.

I did investigate being able to cancel depsgraph evaluation a bit in the past, but didn’t work on it recently.

@rbx775

Will we be able todo nested loops?

Yes.

@rbx775

Will parallel loops ‘just’ be a performance upgrade to serial loops?

No, parallel loop will be a different kind of loop and not just a minor change to serial loops. Parallel loops need a way to reduce the result from the individual loop iterations into one value.

@Bobo_The_Imp

I think Output previous iteration isn’t bad, but perhaps “Cumulative Output” or “Merge Each Iteration” would be slightly better.

Those don’t seem to describe the behavior well I think.

@rbx775

What do you think of ‘Pre-evaluation’ or ‘Check pre-execution’ ?

Those are somewhat better, but still don’t feel right. Maybe something like “Break First”, not sure.

[Bobo_The_Imp]

@jacqueslucke

Those don’t seem to describe the behavior well I think.

Oh yep, sorry I actually jumped to a conclusion sorry. What I was going on about was a function in the loop that allows to join every iterations result into 1 at the end. I saw “output previous” and put two and two together, that’s my bad.

And while I know this isn’t feature request, I think the serial loop should probably have a function like that. It’s not always used in simulations but often enough in procedural modelling. UI wise maybe a checkbox or drop down to choose the behavior.

[HooglyBoogly]

@Bobo_The_Imp

And while I know this isn’t feature request, I think the serial loop should probably have a function like that

That’s probably something better oriented to the parallel loops or lists in the future in my opinion. You can probably do that with this proposal already with the join geometry node anyway. But don’t expect it to be fast!

[BD3D]

Awesome! Is there a way to get the current iteration integer within the loop with this design

[Bobo_The_Imp]

Yes I was going to bring this up with a mock-up later, but I’m too busy atm. But accessing the iteration number to can work some real wizardry on things. Allowing different values for things on each iteration can make complex things very fast.

[HooglyBoogly]

This is possible by adding an integer item in the loop and add 1 in every iteration.

[LazyDodo]

This is possible by adding an integer item in the loop and add 1 in every iteration.

That’s just building a for loop your self. If i have to keep track of the iteration count my self, I may as well add that one compare node in the end and do all of the work to make a for loop.

With the various loop styles, i’m starting to suspect the proposed node is too complicated and the “helping” that we are trying to do isn’t really helping as much and is just getting in the way.

I feel it may almost be better just to have something like this as a base building blocks

Enter determines if the loop is entered, if false the input Geometry is passed through to the output
Exit determines if the loop is exited

enter and exit are not as ambiguous as break and output previous and it allows people to build

• for style loops
• do { .. } while(x) loops
• while (x) { ... } loops (somewhat clumsy since you have to implement the test twice, both for enter and exit conditions)

sure it be easy to make an infinite loop but i feel max_iterations was just a band-aid at best for a that problem (to be solved outside of this proposal) anyhow. It’s only helping me half way with a for loop, it doesn’t protect for the locking up blender problem, so … why does it deserve to be there? It seemingly brings very little to the table.

[Rusculleda]

It may be a language thing but I don’t understand your objection to the max iterations socket. It’s very practical for when you don’t need to control dynamically the number of repetitions, which I’d expect to be a very common situation. If you look at the example Jacques showed with the simulation demo, for that situation and many it’s useful to define beforehand how many repetitions you want. Sure, there’s a version of that which keeps repeating until finding a perfect or perfect enough result and then breaks the loop, but many times that’s not needed, and it’s more cumbersome to manually break the loop.

[LazyDodo]

The design I proposed is a generalized solution, any kind of loop can be expressed with it. They are building blocks to be build upon.

With that in place, we can certainly offer some specialization nodes like straight forward for loops that does have max_iterations (and should also have iteration) values. But those are specializations should not be part of the core implementation.

[dfelinto]

I’m still not convinced on the benefit of having an explicit “Output Previous” option there.

Following that mantra of “Making simple things simple, complex things possible” I think we can try just supporting the basic Break first (which still returns the final result of the current loop evaluation).

Leaving the ability to stop before for artists to do at first with a Switch node. If that turns out to be a rather common case, we then can add this feature.

[BD3D]

@HooglyBoogly

This is possible by adding an integer item in the loop and add 1 in every iteration.

How would that work?
I think having an iter int in the “Serial loop input” output would be quite handy (has been done elsewhere)

[Rusculleda]

I don’t see much trouble in doing it manually

[Bobo_The_Imp]

Is it actually the same though just to join? I might be derping pretty hard, but I think there are circumstances where that won’t work properly, like such:

Just an icosphere copying itself onto its own point, then again, silly fractal stuff.
In this case if I join the original geometry with the result of the tree, then duplicate the geometry nodes modifier (the current loop manual workaround) I will get nasty intersection on the inside:

Because naturally the next iteration has the original geometry and the whole tree, so the instance on points node will produce more copies than really wanted, ideally you just want the copies only from the instanced stuff on each iteration, only “growing” the stuff on the outside, and nothing on the inside pilling up. The wording here is is sort of hard to follow so as reference the desired look is more like this:

This is possible like so:

The instance on points node just leads directly to a new instance on points node, never touching the original input geometry, then you just join it each time with the output. Gah, this is tough to read but hopefully easier to see just following the pictures.

Anyways this works, but it’s no loop-able by duplicating the geometry node modifier right? It needs to be done in the same graph, unless Blender could have multiple geometry streams connected into the Group Output node so they could individually carry over. That’s why an option like “join each iteration” is needed, otherwise such operations wouldn’t possible, at least not through convenient means.

I could just be a big dumb dumb, I’m pretty tired from overwork, but I think this pans out, correct?