2022-06-16 Pipeline, Assets & I/O Meeting

Meetings
1

Everyone is welcome on the Google Meet linked below every other Thursday, at 17:00 CEST.

Present: Aras Pranckevicius, Bastien Montagne, Brian Savery, Dan Englesson, David Ganter, Julian Severin, Michael Kowalski, Sonny Campbell, Sybren Stüvel

The meeting covers the pipeline & I/O module in a broader sense, including some topics hosted in other modules (e.g. some I/O python add-ons, or overrides and .blend file I/O from the Core module).

Links

USD

Michael still has some task/patches to review and push, also working on extending testing for USD.

Brian presents an add-on developped by AMD for USD Hydra rendering and USD composition tools.

Some main points of this tool are:

  • Taking advantage of Hydra, which can merge data from many different sources (like directly from USD, from Blender, etc.) to produce the scene for rendering.
  • Using MaterialX as exchange format for shading.
  • Can in theory be used to integrate Blender into an eterogenous pipeline, can use any Hydra-compatible render engine.
  • Can act as a USD scene compositing tool, using a custom node tree system, and some hacks in Blender data system to get a visualization in the Outliner.

It would need more underlying bricks to be implemented in Blender itself (see e.g. the design task for collection for import/export).
Currently everything is in Python, the project needs to be split in smaller parts for the final design.

The work done by Luma Picture in their USD tools could be used as a reference for the Outliner design?

Python support in USD would be very important there.

Issues with USD libraries (should be dynamically linked, such that they are also shared between all USD tools, also needed for USD plugins, etc.) are also still being worked on.

Overrides

Bastien mostly notes that there has been some meeting regarding overrides to identify important issues from a user experience perspective.

Asset Browser

Julian says that there is still lots of work to be done, progress has been slow recently. More help from the community would also be great.

Other Topics

Aras would like to remove the ‘experimental’ tag on the new C++ OBJ IO code. The Exporter seems OK, importer is more recent and status is not so clear, but so far it seems in good shape too.

Plan is to remove experimental flag soon (and then tag the Python add-on as ‘deprecated’), and re-assess status before 3.3 release.

Aras also has a few patches waiting for review, that greatly improve some import operations speed.

Next Meeting

The next meeting will be on Thursday 30th June, 17:00 CEST/Amsterdam time (your local time: 2022-06-30T15:00:00Z).

The provisional meeting agenda will be linked in the #pipeline-assets-io-module channel before the meeting.

7 Likes
2

@mont29 as mentioned in the other thread can we present on our add-on for USD Hydra rendering and USD composition tools? Please add to the agenda!

3 Likes
3

@bsavery ah right, done! :slight_smile:

1 Like
4

Hi all,

Great to have news on USD :slight_smile:
@bsavery amazing work on the add-on, looking forward to trying it out.
Could someone quickly go over how memory is managed when rendering through USD compared to Cycles ?

There was also some talk a while back about how having Hydra render delegate support might open the door to some memory improvments. Would this be viable ?
Could be a big help when dealing with large scenes.

5

Good questions.

And I think both can be answered in the same response. Currently when you are linking a blender scene or importing a USD into Blender, all the data is loaded into Blender’s memory. When you go hit render, everything is exported to whatever renderer you are using, EEVEE, Cycles, ProRender, etc.

However, the way USD and Hydra works, you can have a system such that it is loading the USD structure and only what is needed for Blender to work. The actual geometry etc only needs to sent to there renderer for the viewport or final render, so doesn’t have to live in Blender’s memory. This can lead to faster and less memory intensive rendering of references or linking. This is pretty much exactly what we did, and propose to include with Blender built in. It might not be needed for many users, but a huge improvement for studios.

3 Likes
6

Out of curiosity / just a thought.

Would it make more sense to include nodal USD tools into Geometry Nodes instead of a separate context?

I understand the data flow and types are different but the ability to orchestrate and manipulate USD data directly inside Geometry nodes seems like a much more flexible approach and would probably consolidate development efforts.

1 Like
7

Thank you for these insights Brian !

The actual geometry etc only needs to sent to there renderer for the viewport or final render, so doesn’t have to live in Blender’s memory.

It might not be needed for many users, but a huge improvement for studios.

What would prevent this from working as a generalised solution ? i.e. any time the scene is bigger that half the available RAM, it is exported to USD, offloaded from Blender and rendered through Hydra. Couldn’t this basically double Blender’s RAM cap ?

@jason-apple @Michael-Jones-Apple With M1’s unified memory architecture, couldn’t this even triple the memory cap if the scene can be directly read from the memory pool ?

8

I’m not sure I see that exactly. For example there is a reason that shader node trees are separate from geometry nodes. They are as you said different contexts for a reason. And I think USD nodes fall into the same category. Different things where the USD nodes are more about manipulating the structure of USD data stream. Furthermore while both would be used independently usually.

But if @sybren or @mont29 or someone from the Blender side feel differently I’d be open to the discussion.

9

Well, either the user or Blender would have to choose what to offload which doesn’t have an obvious easy answer. Most users who are working on scenes that big are breaking them up anyway.

10

Here’s a draft of what we’re going to submit as a design task on developer.blender.org. Feel free to comment.

Description: An extension of current USD workflows in Blender and enabling studio pipeline workflows for USD. This would enable using the USD Hydra rendering system as well as allowing references of USD assets for rendering without loading them into Blender. Furthermore exporting of material nodegraphs via MaterialX is added.Hydra - is USD’s rendering system. It allows “multiple heads” (hence the Hydra name) which means you can feed data in from multiple sources.
MaterialX - Academy Software standard for material interchange. Supported by Autodesk, AMD, Pixar, etc. And fully supported in USD for abstracting materials across renderers.

Big picture:

  • Adds similar tools to Blender like Houdini’s Solaris or Maya’s USD plugins. The benefits are:
  • Better support for rendering addons. Addons can reduce boilerplate code by supporting rendering through Hydra
  • Complex material graphs are supported for import and export, not just through USD Preview surface.
  • USD data can be referenced without loading fully until render time
  • Addons and scripts can use the USD Python api

Use cases:
Users wanting to load, manipulate and composite USD data in Blender without destructively importing the USD.
Users wanting to interchange material networks via USD/MaterialX with other software.
Renderers wanting to support Blender without writing a whole addon.

Design:
This project has many sub tasks so we will break them up and discuss.

  1. Add ability for Blender to dynamically load a USD library. This will be using the USD/BUILDING.md at release · PixarAnimationStudios/USD · GitHub internal monolothic library. This is needed because static libs cannot create python bindings or load plugins.

  2. Expose the USD python bindings and USD plugin system. Necessary for manipulating USD in python and loading rendering plugins.

  3. Add a “Hydra Rendering Addon” with python and C parts like EEVEE and Cycles.

    a. Python parts of hydra rendering addon - setup render UI and launch render through C. Will search for installed Hydra render delegates (default to the built in OpenGL, maybe include Cycles delegate). Will provide a way and documentation for other renderers to install render delegates.

    b. C part of rendering: Loads a USD stream and renders passing the rendered frame buffer back to Blender. The USD stream is selectable to come from either:
    1. Solely from Blender data. Blender data is exported to USD via a USD Scene Delegate: Universal Scene Description: HdSceneDelegate Class Reference
    2. A USD nodegraph which composites the USD data. Note that in the USD nodegraph pieces or all of the Blender data can be read in as well.

  4. USD nodetree system - A python based custom nodetree. Nodes are for compositing and manipulating USD data and written in python with a compute() method. These use the USD python binding. A nodetree can be used for exporting or rendering through the Hydra addon.

  5. When the Hydra addon and USD nodetree are selected, show a “USD Collection” in the outliner which is a tree view of the USD data. This would allow selection and manipulation of items in the USD stream via the blender ui for:

    • moving, scaling, rotating
    • assigning materials
    • visibility, etc

This would involve some deeper changes sketched out in ⚓ T68933 Collections for Import/Export to allow empty typed objects which link to a USD prim entity.

  1. A Material Exporter system which allows exporting shader nodegraphs into the USD stream. Due to the nature of USD, multiple nodegraphs can be exported:

    1. Cycles nodegraph - for re-importing with no data loss back into blender. The current exporter puts everything into USD preview surface
    2. MaterialX nodegraph - translates cycles nodes to MaterialX
    3. Possibility for extending to other material exporters
6 Likes
11

As platform dev facilitating most deps, may i ask: Can the pipeline team for the sake of my sanity please get together and make up their minds on USD static or dynamic? Last time this came up, I spend quite a bit of time getting USD dynamic (inc the py bindings on windows), which got reverted since the admins wanted to stay static (for linux+mac reasons), and we’d have to look into if we can make the python bindings work with static USD. I have since spend quite a bit of time on this (not quite at the finish line, but it looks doable) and now we’re back to wanting dynamic.

I’m happy to facilitate whatever the pipeline team/admins decide on, but pick something!!

1 Like
12

Can you point me to that previous conversation?

1 Like
13

I can’t seem to dig up anything in my inbox, @brecht do you have any records of this? I checked the 3.2 lib update diff and branch, but nothing stuck out there either maybe it was discussed on chat?

1 Like
14

Regardless of the previous conversations, including USD as a static library precludes having USD plugins (including Hydra rendering delegates) and Python bindings of USD.

If this is the way BF is determined to go here, all the other conversation is moot.

1 Like
15

This we can likely make work statically, one of the kraken devs managed to get it working for him. I’m currently trying to get there as well, got them building statically and linked into the monolithic static lib yesterday, so seeing “back to dynamic” today is a little demoralizing.

Plugins, haven’t looked at them yet, but if it works like every other plugin system on the planet, it seems unlikely to be super thrilled with static linking.

like i said, i have no horse in this race and will make whatever we want work, it just can’t be something different every time i sink time in (cause lets be honest, USD isn’t the fastest dep to build…)

1 Like
16

The discussion I remember about this is here:

I don’t remember there being a decision one way or the other. Probably switching to dynamic libraries is inevitable for Hydra plug-ins support, but this is likely a few weeks of work for platform maintainers or contributors that want to help with this, to deal with the various issues. So time for that would have to be allocated as part of this project.

1 Like
17

I really depends a bit on how many deps they want to drag in, if it’s just USD being dynamic, it’ll be time consuming but nothing in comparison to USD taking on the OSL/OIIO+friends/OCIO/OpenVDB deps which are currently all static as well for us.

If that’s the case we’ll either have to deal with massive duplication between the blender binary and the USD shared lib binary (with TBB being a wild card here, tbb static with multiple instances could be tricky to manage) or start making all our other deps dynamic which… oof… it surely can be done… but…

1 Like
18

If you want to connect with me in PM or email we should discuss this more in depth than is probably making sense for this thread. I haven’t seen that Kraken “wabi” python binding of USD before but it looks like they are maintaining their own USD binding outside of the USD repo. We’ll see if that’s maintainable.

Exactly, I think it hypothetically should be possible to build USD against the same static libs that Blender is using and just load USD libs dynamically. Like you said TBB might be the wild card here. This might take some symbol manipulation on the USD libs.

Furthermore, the “binary add-on” point is interesting to bring up. What this would sort of mean is that any addons using the USD library would have to link against the Blender one as well. And again, this would open up for many more python USD addons…

1 Like
19

Sure thing!

I’m not following their work directly, but given they’ve proven it’s possible, i’m doing changes to the USD build system to facilitate. that being said, if we’re going dynamic, all of it is moot .

Exactly, I think it hypothetically should be possible to build USD against the same static libs that Blender is using and just load USD libs dynamically.

I can take out the hypothetically the 3.2 libs were like this for a bit on windows (at least locally for me) it will definitely work no need for symbol manipulation even, but you end up duplicating the static deps in the USD binary, their size is not insignificant, so it’s an undesirable option rather than a not possible option.

USD dynamically picking up on the static libs inside the blender binary, i can’t even see hypothetically working.

1 Like
20

I think we’d make USD and its direct dependencies all dynamic libraries. For Hydra you want to be able to exchange OpenVDB grids and share OpenColorIO configuration, which I don’t think works when duplicating them as static libraries inside the USD library.

We probably do not want to go all the way making every library dynamic, to avoid symbol conflicts. LLVM in particular is problematic with graphics/compute drivers. And for example image file format libraries do not have versions specified by the VFX platform.

1 Like