MiLiAN GPU Simulation framework. Framework available, integration with Blender?

That is 50% of the question :slight_smile: Plugin or core, either way, how? Or do you mean a stand-alone and then a plugin to communicate with blender? Please elaborate.

something like this maybe? The Grove 2.0 - The Grove , see the part about portability. Although it says it is in rust, I am mainly referring to the approach.

2 Likes

I think this leaves me where I started. Thanks for the link thou, looks like a great system. My code could potentially interface with any host. Let me illustrate.

If “you” want a tight integration, blender must provide (or get):

  1. A capable, scalable particle system with no fear of HUGE point clouds (includes geometry)
  2. A capable, scalable volume system with no fear of HUGE voxel grids (sparse or dense, irrelevant)
  3. Tools for processing such data in creative ways without crashing or freezing or dos of any kind.
  4. Most importantly: A way for me to read and write this data via an api (or directly, if core code). Core implementation would potentially avoid massive redundancies, data wise.

If blender does NOT have this, then we are on a plugin path:

a. I must use my own particle system
b. I must use my own volume system
c. I must build my own tools for post processing
d. I still need that api, but now I only share data relevant for visualization and all operations are in my code, CPU or GPU. More memory efficient, but this would also mean that file export (baking) does not go through blender. It would also, by definition, limit the degree of integration with blender.

Exchanging data is my main concern because I can see that blender does not “want” me to do 300mil “particles” anywhere. So yes, a plugin seems to be the way to go. But again, I lack experience as blender dev and thus ask for advice and potential pitfalls.

It is also possible that the time is just not right, regardless of past achievements in development. Maxons Cinema 4D took about ~15 years to even get started on a unified simulation system and nodes + a serious update of their core. For similar reasons. Even today, C4Ds USD support is lacking and performance with point clouds and particles is years behind others. If blender as a product or community has no true desire to focus (and I mean focus) on this area, then plugins will always be the “solution”. Not a deal breaker, but it changes the game considerably, I think (from a long term dev-perspective).

8 Likes

A significant pitfall is not only the lack of a C++ API (which of course you’ve already noted), but that the Blender Foundation is categorically opposed to offering one, full stop. This has been stated on more than one occasion in the past, and I’ve seen no public discussion of this changing (nor even a desire for it to change).

(I agree that such a tool would be very advantageous for your goal; I’m just offering an FYI to the reality.)

2 Likes

Thank you, yes I have read about such requests and their denial a lot in the past years, but I did not understand it as a complete “NEVER”. Getting closer to that now :wink: Not a problem, if the python api is equally performant. Which it clearly is not, based on my research so far. So either the goal is to have stuff all in core, or simply not to care. Or it is “simply” a matter of protecting the devs from complete overload?

Cannot pretend to know better at this point, but the reasoning behind such a decision would be great. After all, blender is written in C/C++, if I am not mistaken.

A representative thread of a discussion that included both users and Blender developers is here. The whole thread isn’t about that, but a significant part of it is. Note that the situation isn’t quite as thorn-neverwake describes: if you look at this post you will see support from a key Blender developer for the general idea, with the main reason cited for not having it being that it is a lot of work to do right.

4 Likes

My impression has been that Ton does not support the idea of an API; he even addressed it directly in October at BCON. It didn’t sound like “good idea, but lots of work” - rather, “no, this is not what we want for Blender.”

Of course, I don’t know Ton personally nor work for Blender. So perhaps my impression is wrong; I do hope it is wrong.

1 Like

I’m not a spokesperson of the foundation, just a developer, but to me it always felt pretty intuitive as to why there isn’t a C++ API. The mission of the Blender Foundation:

“Get the world’s best 3D CG technology in the hands of artists as free/open source software.”

Now think about what a C++ API might bring to Blender. Maybe bigger studios might use Blender, maybe not. Maybe we’ll have new UV tools, new high-performance simulation tools, mesh-generating AI tech… but all sold under a commercial license. We’ll probably have to deal with a whole bunch more bugs. We’ll probably have to be more careful than ever about breaking the API. I think you get the idea. All of that doesn’t help the mission. The mission isn’t to create a marketplace for closed-source addons. The mission isn’t to spend time developing and maintaining an API that won’t help us accomplish the goal of “the freedom to create”.

TL;DR: IMO it’s unlikely we’ll have a C++ API because it’s a lot of work that does not align well with the mission.

17 Likes

So far, so good. Or not. I read through that post, but since it is literally years old, it may no longer be entirely representative? I understand the concerns regarding C vs C++ and platform compatibility / support, but I find no hard reason why this is “not going to happen”. Frankly, to me it smells like fear of too much work because:

  1. Old code base. You will find issues you do not want to find, and have to fix them first (or work around).
  2. Breaking old stuff in service of new exposed api to SDK. Not ideal, even more work.
  3. Too much to do already, just to keep the train going. Wherever it may go, it must go.
  4. Add-Ons will “most of the time” be an okay solution, so why bother.

If this is the essence of it and I am not missing anything, I would risk the statement that blender is not scalable enough to survive the next 10 years with this process, since others (like Houdini) are providing very attractive licensing models and will continue to do so, the more market share they gain (Houdini specifically).

But then again, I cannot know. Just a feeling.

And I also stumbled upon a few disappointed mac users :wink: Understandably so, but the question of “what do we want, why and how” seems to be very pressing already, without a stranger poking around.

Edit: another thought. Looking at the demos, tutorials and medium to larger scale use cases of blender (in blender context), it seems clear that blender is on a similar road as Maxon was or is: The desires of the user base are perhaps not evaluated enough against realistic outlooks and future technologies. Again, this is a feeling for now. It might just be my blender-rookie ignorance. I hope so.

1 Like

But what is “the best”? Is that not at the core of the issue? I am not the one to define that, but I must ask. Curiosity demands it!

Its not the “best” part you should be focusing about, but the part about getting it in the hands of artists for free. Yes C++ API will greatly expand Blenders capabilities, but if people who create c++ plugins are selling them to few who can afford, or simply not sharing, who’s benefiting from those capabilities? Is it getting in the “hands of the artists for free?”

1 Like

The people who purchase them benefit.

Just as when you pay for your lunch… who benefits? You, and the grocery store.

Or the computer, that someone bought to RUN the software. Did everyone get a free computer, or did both the buyer and seller benefit from the transaction?

1 Like

And do you get the impression that Blender is focused on people who are purchasing 3D softwares and plugins? Again, refer back to statement about the mission.

2 Likes

Nick, you asked, “who would benefit”. I answered the question, and the answer isn’t an opinion - it’s a factual result of paying for something, and getting what you bought. I’m sure you have bought SOMETHING in your life. And felt that it was a benefit. I’m sure you have sold your time to someone, and they paid for your time, because they wanted a benefit.

I plainly stated earlier that I believe a C++ API isn’t coming, due to Ton’s feelings on open source and the mission. Please don’t re-frame my answer into an argument that I’m not making.

1 Like

Well, thanks then, seems like interesting contribution to conversation!

1 Like

I do for free what I can afford. Focus on free means “don’t ask too many questions, it is free”. I do not support that sentiment. I also do not believe that the existence of a C++ api or any SDK is inherently tied to expensive plugins or an ecosystem where blender is “less free”. Makes no sense to me. Coding takes time, developing any complex product is a risk .

To stay on topic, the question about “best” was a serious one, but I realize this may not be the right place. After all, this threads headline is a different one. I asked because I can read a mission statement, yet not understand the mission in detail, as the statement is in fact vague, considering the times.

If I could finance development all through patreon or similar, then I would probably not care about selling licenses. But can I? Should I? Am I not selling all the same, just differently? I get your idea about free blender + free extensions, but at some point, contributions must be respected as what they are: Work. And if we want “the best” work, all the time, we must at least consider to make it easier or pay for it. As does the blender foundation, so no problem there.

Edit:
Your point about “who blender is focused on” is actually part of what I am after, it may help to understand. If you care to elaborate your point of view, please do.

5 Likes

UVPackmaster would be a good thing to look into. It’s a paid, proprietary C++ engine with a free open-source Numpy bridge into Blender. They have tens of thousands of customers, there’s a large market (myself included) that is willing to pay a lot of money for good Blender plugins

3 Likes

On that point, while I don’t own it, I would have thought that the Flip Fluids addon at it’s core is C++ and the result just gets feed back into Blender via Python?

I mean all of that simulation would be slow as hell if the whole thing was being done just using Python and the standard Blender API, wouldn’t it?

That is indeed the case!

@ Anthony_Gibbs:
Who said otherwise? Perhaps I should ask the developers of the FLIP addon for advice or experience …