And I think both can be answered in the same response. Currently when you are linking a blender scene or importing a USD into Blender, all the data is loaded into Blender’s memory. When you go hit render, everything is exported to whatever renderer you are using, EEVEE, Cycles, ProRender, etc.
However, the way USD and Hydra works, you can have a system such that it is loading the USD structure and only what is needed for Blender to work. The actual geometry etc only needs to sent to there renderer for the viewport or final render, so doesn’t have to live in Blender’s memory. This can lead to faster and less memory intensive rendering of references or linking. This is pretty much exactly what we did, and propose to include with Blender built in. It might not be needed for many users, but a huge improvement for studios.
Would it make more sense to include nodal USD tools into Geometry Nodes instead of a separate context?
I understand the data flow and types are different but the ability to orchestrate and manipulate USD data directly inside Geometry nodes seems like a much more flexible approach and would probably consolidate development efforts.
The actual geometry etc only needs to sent to there renderer for the viewport or final render, so doesn’t have to live in Blender’s memory.
It might not be needed for many users, but a huge improvement for studios.
What would prevent this from working as a generalised solution ? i.e. any time the scene is bigger that half the available RAM, it is exported to USD, offloaded from Blender and rendered through Hydra. Couldn’t this basically double Blender’s RAM cap ?
@jason-apple@Michael-Jones-Apple With M1’s unified memory architecture, couldn’t this even triple the memory cap if the scene can be directly read from the memory pool ?
I’m not sure I see that exactly. For example there is a reason that shader node trees are separate from geometry nodes. They are as you said different contexts for a reason. And I think USD nodes fall into the same category. Different things where the USD nodes are more about manipulating the structure of USD data stream. Furthermore while both would be used independently usually.
But if @sybren or @mont29 or someone from the Blender side feel differently I’d be open to the discussion.
Well, either the user or Blender would have to choose what to offload which doesn’t have an obvious easy answer. Most users who are working on scenes that big are breaking them up anyway.
Here’s a draft of what we’re going to submit as a design task on developer.blender.org. Feel free to comment.
Description: An extension of current USD workflows in Blender and enabling studio pipeline workflows for USD. This would enable using the USD Hydra rendering system as well as allowing references of USD assets for rendering without loading them into Blender. Furthermore exporting of material nodegraphs via MaterialX is added.Hydra - is USD’s rendering system. It allows “multiple heads” (hence the Hydra name) which means you can feed data in from multiple sources.
MaterialX - Academy Software standard for material interchange. Supported by Autodesk, AMD, Pixar, etc. And fully supported in USD for abstracting materials across renderers.
Big picture:
Adds similar tools to Blender like Houdini’s Solaris or Maya’s USD plugins. The benefits are:
Better support for rendering addons. Addons can reduce boilerplate code by supporting rendering through Hydra
Complex material graphs are supported for import and export, not just through USD Preview surface.
USD data can be referenced without loading fully until render time
Addons and scripts can use the USD Python api
Use cases:
Users wanting to load, manipulate and composite USD data in Blender without destructively importing the USD.
Users wanting to interchange material networks via USD/MaterialX with other software.
Renderers wanting to support Blender without writing a whole addon.
Design:
This project has many sub tasks so we will break them up and discuss.
Add ability for Blender to dynamically load a USD library. This will be using the USD/BUILDING.md at release · PixarAnimationStudios/USD · GitHub internal monolothic library. This is needed because static libs cannot create python bindings or load plugins.
Expose the USD python bindings and USD plugin system. Necessary for manipulating USD in python and loading rendering plugins.
Add a “Hydra Rendering Addon” with python and C parts like EEVEE and Cycles.
a. Python parts of hydra rendering addon - setup render UI and launch render through C. Will search for installed Hydra render delegates (default to the built in OpenGL, maybe include Cycles delegate). Will provide a way and documentation for other renderers to install render delegates.
b. C part of rendering: Loads a USD stream and renders passing the rendered frame buffer back to Blender. The USD stream is selectable to come from either:
1. Solely from Blender data. Blender data is exported to USD via a USD Scene Delegate: Universal Scene Description: HdSceneDelegate Class Reference
2. A USD nodegraph which composites the USD data. Note that in the USD nodegraph pieces or all of the Blender data can be read in as well.
USD nodetree system - A python based custom nodetree. Nodes are for compositing and manipulating USD data and written in python with a compute() method. These use the USD python binding. A nodetree can be used for exporting or rendering through the Hydra addon.
When the Hydra addon and USD nodetree are selected, show a “USD Collection” in the outliner which is a tree view of the USD data. This would allow selection and manipulation of items in the USD stream via the blender ui for:
moving, scaling, rotating
assigning materials
visibility, etc
This would involve some deeper changes sketched out in ⚓ T68933 Collections for Import/Export to allow empty typed objects which link to a USD prim entity.
A Material Exporter system which allows exporting shader nodegraphs into the USD stream. Due to the nature of USD, multiple nodegraphs can be exported:
Cycles nodegraph - for re-importing with no data loss back into blender. The current exporter puts everything into USD preview surface
MaterialX nodegraph - translates cycles nodes to MaterialX
Possibility for extending to other material exporters
As platform dev facilitating most deps, may i ask: Can the pipeline team for the sake of my sanity please get together and make up their minds on USD static or dynamic? Last time this came up, I spend quite a bit of time getting USD dynamic (inc the py bindings on windows), which got reverted since the admins wanted to stay static (for linux+mac reasons), and we’d have to look into if we can make the python bindings work with static USD. I have since spend quite a bit of time on this (not quite at the finish line, but it looks doable) and now we’re back to wanting dynamic.
I’m happy to facilitate whatever the pipeline team/admins decide on, but pick something!!
I can’t seem to dig up anything in my inbox, @brecht do you have any records of this? I checked the 3.2 lib update diff and branch, but nothing stuck out there either maybe it was discussed on chat?
Regardless of the previous conversations, including USD as a static library precludes having USD plugins (including Hydra rendering delegates) and Python bindings of USD.
If this is the way BF is determined to go here, all the other conversation is moot.
This we can likely make work statically, one of the kraken devs managed to get it working for him. I’m currently trying to get there as well, got them building statically and linked into the monolithic static lib yesterday, so seeing “back to dynamic” today is a little demoralizing.
Plugins, haven’t looked at them yet, but if it works like every other plugin system on the planet, it seems unlikely to be super thrilled with static linking.
like i said, i have no horse in this race and will make whatever we want work, it just can’t be something different every time i sink time in (cause lets be honest, USD isn’t the fastest dep to build…)
I don’t remember there being a decision one way or the other. Probably switching to dynamic libraries is inevitable for Hydra plug-ins support, but this is likely a few weeks of work for platform maintainers or contributors that want to help with this, to deal with the various issues. So time for that would have to be allocated as part of this project.
I really depends a bit on how many deps they want to drag in, if it’s just USD being dynamic, it’ll be time consuming but nothing in comparison to USD taking on the OSL/OIIO+friends/OCIO/OpenVDB deps which are currently all static as well for us.
If that’s the case we’ll either have to deal with massive duplication between the blender binary and the USD shared lib binary (with TBB being a wild card here, tbb static with multiple instances could be tricky to manage) or start making all our other deps dynamic which… oof… it surely can be done… but…
If you want to connect with me in PM or email we should discuss this more in depth than is probably making sense for this thread. I haven’t seen that Kraken “wabi” python binding of USD before but it looks like they are maintaining their own USD binding outside of the USD repo. We’ll see if that’s maintainable.
Exactly, I think it hypothetically should be possible to build USD against the same static libs that Blender is using and just load USD libs dynamically. Like you said TBB might be the wild card here. This might take some symbol manipulation on the USD libs.
Furthermore, the “binary add-on” point is interesting to bring up. What this would sort of mean is that any addons using the USD library would have to link against the Blender one as well. And again, this would open up for many more python USD addons…
I’m not following their work directly, but given they’ve proven it’s possible, i’m doing changes to the USD build system to facilitate. that being said, if we’re going dynamic, all of it is moot .
Exactly, I think it hypothetically should be possible to build USD against the same static libs that Blender is using and just load USD libs dynamically.
I can take out the hypothetically the 3.2 libs were like this for a bit on windows (at least locally for me) it will definitely work no need for symbol manipulation even, but you end up duplicating the static deps in the USD binary, their size is not insignificant, so it’s an undesirable option rather than a not possible option.
USD dynamically picking up on the static libs inside the blender binary, i can’t even see hypothetically working.
I think we’d make USD and its direct dependencies all dynamic libraries. For Hydra you want to be able to exchange OpenVDB grids and share OpenColorIO configuration, which I don’t think works when duplicating them as static libraries inside the USD library.
We probably do not want to go all the way making every library dynamic, to avoid symbol conflicts. LLVM in particular is problematic with graphics/compute drivers. And for example image file format libraries do not have versions specified by the VFX platform.
I’m not familiar enough with USD to determine what deps are must haves and what are nice to haves, pretty sure brian may have some insights there.
Just to be sure, going foward, USD + select (TBD) deps dynamic for all platforms? ( I can manage windows+linux, but if linux works pretty sure you’ll be able to manage mac) if so i’ll work with brian to get atleast the library part taken care of.
Yes, I think we should make at least USD, OpenVDB, OpenColorIO and TBB dynamic for all platforms. There may be some things for binary add-on authors to deal with, but it’s the same for other 3D apps. We probably need to make some tweaks and document the solutions for avoiding conflicts.
Sergey also agreed going to dynamic libs is what we need to do.
TBB is a must & so is Boost – Regarding OpenVDB / OCIO - OpenVDB is all silo’d off into it’s own plugin/library and OCIO is pretty specific to roughly ~4 lines of code in just hdx if I’m not mistaken, it’s some very basic support for supporting color management configs.
I personally just ended up going fully static here - but there is “more work” involved at managing dependencies (py dependencies on each other are cyclic and would need to be stopped from Tf manually loading them in – along with most of Tf’s library loading ability can/should be disabled if those resources are managed within a static build context – and since Pixar didn’t spec any of this out for a static build - Tf pretty much always assumes it’s comfortably within a dynamic context.)
Also - yes my binary size blew up. Currently it’s sitting at half a gig now - which I think is up at least twice to what it used to be.
Also, effectively 90% of TBB is wrapped to one central place in all of USD - you’re going to want to look at WorkDispatcher::WorkDispatcher(), it is here that Pixar has centralized their dependency on TBB - as it is in charge of threading all the templated Run() & Cancel() calls throughout USD, accompanied by a Wait() until those tasks complete. So that’s a plus.
For those familiar with TBB, it’s simply a tbb::task_group for each task context, which deduces a callable “task” to execute → ()().
