Blender's architecture concerning everything nodes

Archive User Feedback
, ,
1

Blender’s architecture

Hello guys,

After some ‘heated’ discussions on the other thread, the one about the weld modifier, I took the liberty to come up with some improvement ideas concerning the upcoming everything nodes features.

First, just a bit about myself: I am a SW engineer myself with over 20+ years of experience including knowledge of C++/Obj-C/Java/JavaScript, now Python and may other things I don’t even remember. Also, I have some experience with modo and houdini. I don’t have an website, but if you curious, you can check some stuff I did in houdini here: Skeleton curve implementation based on VDB medial axis | Forums | SideFX

Also, this doc is meant exclusively to raise awareness, not a demand or intention to be inconsiderate to the dev team, like someone suggested in the other thread.

I am not sure if any of these subjects have been touched by the docs or discussed before, but I am aware of the road-map concerning everything nodes: Source/Nodes/EverythingNodes - Blender Developer Wiki, more here Source/Nodes - Blender Developer Wiki

As you can see above, everything nodes it’s a ‘grassroots’ project. It’s a rewrite of everything based on nodes and these low level ‘functions’ which are acting as low level execution units.
Now, I think I know where this is coming from. These functions are easy to compile/transpile and execute on CPU/GPU LLVM and other platforms. Also, there is the intention to integrate these
functions with other parts in blender. Everything nodes is designed with optimizations in mind.

Ok, here is my main concern with this approach. It sounds good on paper, but in reality things will not stay the same. Why I am saying this?

If you analyze the current state of affairs in blender you will immediately see some concerns:

Language inflexibility

  • The main problem in blender is the fact that is written in C. We know that C is a system language and very strict about everything, strong typing, static dispatch, etc.
    All these things are great when you are writing oses and low level embedded code, but not so great for writing applications: slow dev sprints, lots of segmentation faults, lack of proper
    memory management, slow refactoring, inflexibility in general. What is the solution here? Loosen up. Decouple everything. Dynamic dispatch through unified publish/subscribe bus (like this one https://zeromq.org/), better mem mgr with reference counting, declarative interfaces everywhere if possible through simple definition languages like JSON

EDIT: Dynamic dispatch is intended for exclusively for HI-LEVEL functionality(stuff you usualy expose through plugin interfaces), not for every C function call.

Event system

  • Lack of an unified event system. If you are writing a plugin, the first thing you are going to notice is the fact that you don’t have an unified event bus. you can’t listen to keyboard events
    without writing a ‘modal’ operator. If you want to update state from a draw function, you get a segmentation fault. Everything is in lockdown mode while drawing. Why? You can batch updates
    for later to be executed. Having an unified: UI, timer, draw, input, ops/modifiers/animation life cycle events, for internal or external use, accessible through a common socket based interface is not much of an issue. I think.

Interfaces/contract

  • Lack of an unified interface for ops, modifiers, silent modifiers(like auto merge in edit mode, keyframing/animation/constraints/drivers) and external addons. I believe that most of these can share a common interface based on pub/sub event subscription.
    Splitting them into separate concerns is not exactly following the SoC principle. Breaking them apart into: function, simulation, with binding etc. Data(ram) and time(cpu ticks) are just resources and event triggers. There is no difference between a bevel modifier and fluid simulation. One is data dependant and the other one is data and time dependant, one extra dimension to deal with. But in essence, they are doing the same thing, data transformation. I know that now modifiers are working with DerivedMesh, but switching to the new architecture would be easy when you have proper lifecycle events for operators.
    You can chain them, batch them, wait for parallel finalization, run them on ‘live queries’, and depsgraph can be used to manage the data and make sure it’s staying immutable(I think)
    Operators have the same problem. In reality, there is no SoC between modal/interactive and simple execute operator. They differ only in terms of decorations.

Live events

  • Event masking/filtering/conditionls/livequery. Introducing the concept of ‘live queries’. Live queries are just simple events expressions which can combine multiple sources under a single event subscription. E.g if (keyboard shortcut event && ops.add mesh completed) then show a popup message. This will be essential if you want param expressions for node params like H does it.

Execution

  • Fire and forget, stateless, task based, queued execution units(ops/modifiers/addons/etc) managed through:
  1. zeromq(event bus)
  2. depsgraph(data mgr)
  3. thread & task queue mgr similar to iOS GCD. I don’t know exactly how this would work in C because you don’t have managed mem and closures, but if you come up with declarative input and output definitions for the depsgraph to know what you want to keep alive or not for longer periods of time,
    this might work… also data(objects/mesh) can be bound to the node itself.

Data binding

  • Drivers and constraints in houdini are nothing more nothing less but math expressions with param linking/binding

  • Fully declarative interfaces and event definitions(JSON/protobuf/flatbuffers like) for the above. You can exclude the rendering(EEVEE/Cycles) side, for now, for performance reasons.

  • Node parameter linking & node paths. You should be able to link/bind node params across node groups /networks via node path definition similar to a file path definition.

  • Default params everywhere. convention over configuration here. The concept of ‘sensible defaults’ applies here very well. The user is only modifying the defaults. In this way you make sure that ‘operators’
    are not producing crashes or unstable results.

  • Data with attributes. Deferred responsibility vs micromanagement is the question here. So far, I have not touched this problem at all because, from my pov, it the least important. It’s just data with attributes. The way you look at it is relevant only in context.
    You don’t want to attach more meaning than it’s necessary to the data. In H they have points/primitives/vertex/detail and object attributes with precedence from the lowest to highest, and that’s the whole philosophy.
    Geometry attributes

Plugins

  • Same generic operator pub/sub interface for plugins as well, sockets & shared mem instead of language based(ctypes) language bridging. This will allow for any kind of scripting engine to communicate with blender, including clustering multiple blender ‘nodes’, running in parallel, and cloud support.

UI/UX

  • UI/UX Everything nodes is not going to mean anything, if the end user is not happy :slight_smile: From what I have seen in the docs, these low level ‘function’ execution units are very simple and similar
    to math utility functions. Attaching only one of those to a node is not going to solve a real problem. Like bevel, or boolean modifier does. Of corse, you can chain them, or create sub-networks
    of such low level nodes, but still not very user friendly. The user wants a minimum number of nodes on screen because is hard to follow a large tree of nodes everywhere. What can be done here?
    Well, the node tree should be context aware and pass-through. What this means? It means, that you can connect anything to anything, and the node: based on it’s function, context and parameters
    will apply the transformation only to the part is concerned with. This way, you are going to eliminate lots of unnecessary conversion nodes, and many input nodes if you can pass these inputs as params to the node itself. Exactly, the way houdini is doing it. Or, even better, treat collections, objects, mesh data on the same level. Multilevel is inconvenient sometimes in H.

Edit mode in H is just a single node!! You can have multiple, such nodes chained

TODO

Steps for ‘quick’ refactoring:

  • study zeromq or similar
  • study declarative programming and interfaces
  • study JSON/protobuf/FlatBuffers or similar for serialization across networks, also shared memory
  • integrate zeromq
  • define the new generic ops/modifiers/addons interface
  • prepare the run loops
  • run every event through the new run loops system
  • add support for execution queues(tasks you want execute async in the right ctx)
  • slowly move the current ops/modifiers through the new interface
  • add nodes and new stuff :slight_smile:

Conclusion

Overall, I believe that a proper architecture should be a little bit more courageous in terms of technologies and modern design practices. Switching from basic proceduralism
to a more declarative and functional approach is not such a big deal, but is coming with great advantages in terms of time to market, flexibility, stability and quality overall. Yes, you
might have to sacrifice a little bit of performance. IMO, you don’t want to start with low level optimizations until you have a working PoC with everything else in the picture. I wouldn’t jump the gun here, for such major refactoring. Think about it. What happens to your low level functions if your current data model in blender is not generic enough or performant enough, are you going to rewrite everything again to accommodate the new data model? I believe this matter is being approached from the wrong end… blender, right now, lacks the ‘infrastructure’ to support something so dynamic as ‘nodes’ with interoperability across all systems in mind.

EDIT: This is suppose to be hi-level architectural change. To add a ‘cake layer’ on top of the existing code base. I explained better in the post bellow.

FYI @brecht @ideasman42 @jacqueslucke

7 Likes
2

Merge this here Blender's Architecture (split from merge modifier discussion)

3

I know this looks more like a PPT, and probably some things here don’t really make sense. Eventualy, I can explain, if that’s necessary

4

There are some good ideas here and some (in my opinion) not so good ones. I applaud the long doc with lots of thoughts. I doubt that Blender will adopt most of this since it would be a fairly massive change to a fairly massive codebase representing probably hundreds of person-years of work. But maybe someone would like to try building a new system from scratch using these principles.

I’m not going to respond to everything in the doc, but here are some of my thoughts. I freely admit that these are opinions, not facts, but like you, based on many years (40+) as a software engineer and software engineering manager.

I think there is room for a better and more unified event system, but it would need to be carefully designed: you can’t just have everything hearing and acting on every event, since sometimes events need to be consumed by something that hides those events from other listeners.
The C code does in fact have an event system, but it is not as uniform as you describe, nor exposed all that well to Python API users.

I dislike loose, dynamic coupling. It defers problem to execution time: it is harder to reason about what will arrive where and when. So unexpected things may arrive in unexpected places with unexpected properties that do not obey the preconditions expected by the algorithms and code where they arrive. You can try to say: make everything be able to handle eny garbage thrown at it gracefully, but that is both more tedious to program and doesn’t really solve the problem - the code may not crash, but it certainly won’t do what you are hoping it would do. In some ways it is better to crash, so that developers are made aware of problems and can fix them.

There is a tension between efficiency and generality/ease-of-writing-correct-programs/ease-of-reasoning-about-programs. The system you are describing is likely to be more on the latter side of that tradeoff. If you are more willing to copy-and-mutate rather than mutate-in-place, you are on the latter end of the tradeoff. If you forgo caching and reusing computation, you are on the latter end of the tradeoff. I strongly suspect that if one built a system along the lines you are describing, it would be pleasant to program but would perform like a dog when there are tens of millions of geometry elements. And people are already pretty unhappy with Blender’s performance, when it has been programmed in a style more on the former side of the tradeoff. Not just rendering performance (which you admitted might have to be excluded from this architecture in full glory), but also mesh editing performance.

I think you started wondering about Blender’s architecture by wondering: why would adding a weld modifier possibly cause crashes – it must be the fault of the architecture. But it has very little to do with the architecture: it has to do with the complexity of writing geometric algorithms, especially in the presence of data structures that are there to make editing of meshes efficient. You need to account for all the special cases, and no amount of looser coupling is going to make that fact go away. (Again, sure, you might make the actual crash go away, but the result of the algorithm may silently be crap.)

4 Likes
5

I dare not comment on technical structures and coding,
but fom user experience I would like the nodes to have the possibility of interfacing with custom panels, where once the working structure of a node project has been built, with simple drag and drop of some node values, we can build the interface that we can easy use. it serves to manipulate in a simple and immediate way the project that we built with the nodes.

another user has taken this concept and made a video that explains its function here better.

1 Like
6

Just as a sidenote: the way how (OS) events are all handled on the lowest level in the GHOST layer, transforming them all into the Blender events is very useful. With the one main loop (or pump, however you want to call it) controlling it at least this part is relatively easy to grok and reason about - there are no parts of code that handle OS events outside of what Blender provides, preventing a huge amount of problems caused by custom event handling sprinkled all over the place.

If anything an event system (re-)architecture should be built on the existing preprocessing already done by Blender.

And FWIW any memory management system just tends to make developers lazy. Problems wrt memory will pop up regardless, just in different places, and often infinitely harder to debug if garbage collection happens on a thread on random times.

2 Likes
7

@Howard_Trickey, thanks for your insight.
Ok, what I had in mind, is very similar to what modern browsers are doing today. Hybrid solution. You have a very low level(c++) application, highly optimized, with some declarative decorators on top(HTLM, CSS, JS). Right. By dynamic dispatch I was not thinking about replacing every static C call with dynamic dispatch. That would be suicidal. It is meant for hi-level functionality only. The stuff you usually expose through plugin interfaces. That could be rule of the thumb here.

Unfortunately here, there in not much you can do. If you want performance, you will have to sacrifice more RAM/cache on the altar of programming. There is always a trade off between processing and memory consumption. If you want speed, you must sacrifice either CPU cores or RAM depending on how the algorithm was designed to work. In this case you probably want to cache results for every node, otherwise you will have to re-execute those everytime there is a change and besides that it will break the immutability principle introduced by depsgraph. I believe the idiom which can be applied here to some degree is COW(copy on write) so we don’t copy-and-mutate(if not necessary) and we don’t mutate-in-place either.

Yes, viewport/rendering should not be included by this ‘redesign’ for obvious reasons. Opengl/DX/Vulkan constraints and fast draw events. I don’t know if there much you can do here. Maybe, draw batching, occlusion/masking and interpolation where is possible. But I believe this is already done. Why perf is bad in EEVEE? I don’t know. It could be anything eg: too many context switches(mutex), abusive spinlock, misuse of atomics(like this one rB2c7365aec7d8)

:slight_smile: The fact that you cannot easily add a simple weld modifier is a serious architectural problem. Ok. You can’t convince me here. Data is just data. It matters how you look at it. This is how H is doing it. And I believe they are the gold standard nowadays about this stuff. Simple attribute aggregations with parallelization/map/reduce in mind.

@nokipaike
This is exactly how H is doing it. This stuff is there since ages. And as a contra-example, Blueprints in UE4 are really bad. Everyone is complaining about them. You are better off with C++ then.

@jesterKing
Ok, I don’t know implementation details here about low-level system events, how they work, and how they are being processed in general. I was talking about an universal msg bus meant to service hi-level functionality which usually involves direct user interaction and/or interoperability through plugins. How this universal bus system mirrors low level events, it’s an implementation detail. It’s meant to do more than simple pub/sub/dispatch events. You should be able to use ‘live queries’. To be able to subscribe to math expressions and conditionals involving other potential events and the current state of the system. Otherwise it would be hard to implement proper parametrization and parameter binding - a la Houdini. If low level events are firing too often then you can apply some strategies to interpolate or select only the last one for example.

Garbage collector is good. For real time applications it can be indeed a bottleneck, but even there you can have solutions. What Unity did about this: https://blogs.unity3d.com/2018/11/26/feature-preview-incremental-garbage-collection/
Unfortunately in C you can’t really have a GC. You have to rely on reference counting alone. What I was talking about there is more related to the @persistent decorator you have in python. Sometimes, you have to keep alive objects for longer periods of time than anticipated and for that you need protocol exposed through the new declarative API to make them ‘persistent’. This is the hi-level declarative(JSON like) API, socket based instead of language based I was talking in the first post.

If it was done before? It was. Every modern RTS game is doing something similar in terms of event management and network protocols. For example Starcraft 2

So, once again this ‘re-design’ is not a major refactoring. It is meant to stay on top of the existing code base. Transitioning from classical MVC to something more like MVVM.

8

For high level functionality, the trick is to start thinking like the end-user not like a programmer. How would he do it?!

1 Like
9

OK, so you were only thinking about the high level interconnections. That takes away some of my concerns.

I was misled by you proposing this architecture as a solution to the problem of “can’t add a weld modifier easily”. You say you won’t be convinced by my so I won’t argue further why that isn’t the problem at all with respect to that comment.

1 Like
10

@Howard_Trickey
Ok, now I feel like I owe you an apology and an explanation. I apologize and will explain.
First of all this is not coming from the weld mod discussions. I am working on some pet project and what happened in the weld mod discussion was just synchronicity.
Regarding the problem at hand: here I am talking from the experience I have with Houdini. The main problem you are dealing with, there in H is ‘probability’. There is a probability for some particle to exist
or to do not exist. You have no real handle to this particle, sometimes no way to identify it directly. Then, what can you do? you have to defocus and look at the bigger picture. Do a bbox on the area you are
expecting something to be there and re refocus back to low level. If we are talking about the bevel modifier, you are familiar with: I would do a boolean first and reconnect edges back later like what this guy is trying to do here:
https://blenderartists.org/t/wip-bevel-after-boolean/693072. Instead of looking at it from a 2D perspective, you can increase one more and come up with a volumetric solution.
So, my whole point here is that you should be prepared for ‘the improbable’ to happen. Every operator should be designed with this aspect in mind even if the result is rubbish in less probable situations. You can’t cover all uses cases anyway.
Why is an ‘architectural’ problem? It is because they kept adding new features over and again without looking at the legacy code, without writing system test, integration tests, proper refactoring like the one I wrote about.Most of the features introduces, are half baked. Nobody did any real user testing, and if somebody complained about it on the forum, they shut him down. I understand that this is an open source project with limited resources, but that should be reason even more to make your life as a dev more comfortable.
See, this is where modern technologies really shine. They scale very well, a lot easier to maintain and refactor, less testing, less corner cases you deal with and you can do miracles with a small team…

3 Likes
11

good points. what can you do to improve the architecture?

12

I can’t do anything about this. First, everyone needs to be onboard, and then it is a fulltime commitment for a long period of time.

1 Like
13

I agree with most. I work with Houdini, and do love the architecture of it. But it has more then 20 years of development, and the underlying architecture is there from day one, they “only” improve the way you interact with it, and of course add new features “easily” due to the same. I think, GC isn’t the way, I would much prefer to see Blender slowly move to something like rust that is low level performant and avoids lots of seg faults by default, and also plays nice with the C99 code base.

14

Houdini is indeed designed, to deal with non-deterministic systems, from the start. As for the GC I should have not even bring it to the discussion because it’s not possible to have it in C. Rust can be
indeed a solution, and they have a semi-managed memory model through smart pointers. But to throw another competitor into the picture: swift is also a good candidate since they have ARC(auto-ref-counting)
and ABI compatibility with C.

15

My only concern with Swift, is Apple. While Rust seems “more” open and also has ARC (and a bunch of other primitives to help managing refs/pointers safety) and good a multi-threading story. But I don’t want to start doing language wars here :D.
But again I think this discussion is interesting, I for one love blender way of work (have more then 20+ years on other comercial DCC 3D apps), it’s fast at modelling, something Houdini can’t. So for your idea to work, there would be the need to “construct” some lower level data structures that could be both use for fast editing (or at least converted to bmesh), and at the same time pass as data to any sink that would do something with it and generate, mesh, volume, whatever it needs to be the output. I think doing those “lower” level data on top of the existing code base are possible but there would be too much performance loss, when handing off the data. Have you seen the OFX modifier, it does that at the modifier level to some degree, handing mesh data to a unknown “function” and receives something back.

16

Mozilla is behind rust and Apple is behind swift. And swift is also released under apache licence…
Yes, I have seen the OFX modifier, but in order to create something like that you have to overcome some
issues, related to blender plugin API and also licensing issues. GPL is an extremely harmful licence
towards indie developers in general, but also when it comes to interoperability with other commercial solutions.If you are developer, most of your clients will never alow to open source the code you are selling. So, you always have to be very careful about what licenses you have in your project. Usually, MIT, BSD and apache is ok. Anything else is really bad. I remember that many years ago, when I was working for a corporation, we had an internal training exactly about this problem. They explained to us what licences are allowed and which are not. GPL is like an insecticide which kills bad bugs but also kills the good ones :slight_smile: So, OFX to me sounds like a great idea, but I would be very careful about the legal implications

17

Good, I just wanted to ensure the low-lever bits don’t get mixed into this (:

2 Likes
18

I saw you mentioned my proof-of-concept add-on for streaming data into (and out of) Blender. Just here to say I completed the add-on and made a demo video:
BlendZMQ demo]

The code is now also nicely commented, so it should be much easier to understand what is going on.
In relation to ZeroMQ, the relevant code is here: https://github.com/NumesSanguis/Blender-ZMQ-add-on/blob/master/blendzmq_ops.py#L38-L136

3 Likes
19

Cool. My idea was to do a native integration of zmq, but as a POC, it is good enough to exemplify the potential of such approach.

2 Likes
20

I have always wondered what Blender could have been if it followed an open architecture so it’s more flexible like the majority of DCCs out there especially if everything becomes nodes ,this should be noted at least for the next decade or the big milestone whenever that happens.