2022-3-09 Sculpt/Texture/Paint Module Meeting

Participants: Joe Eagar, Jeroen Bakker, Brecht van Lommel, Julien Kaspar

Agenda

  • Discussing the current state of the 3D brush prototype

Notes

  • 3D Brush prototype

    • Jeroen worked on the rasterizer and storing pixels in PBVH

    • Currently smooth performance with 2 million vertices with an 8k texture

      • Initialising the mesh and texture takes a fraction of a second
      • Overall initialising the texture paining can take up to a minute currently
      • Needs further testing where the performance bottlenecks are
    • Good result for basic draw brush functionality so far

    • Next challenges:

      • Is the aimed performance possible? Probably yes, mostly on higher end systems right now
      • Work further on draw brush first & foremost!
        • Make it feature complete
        • Fully prove that the draw brush works with the current approach
        • Observe performance impact with more added features
        • Only then work on other tools
      • Avoiding artefacts and performance issues for seam bleeding
        • Rasterize the entire image ahead of time?
        • Potentially reuse current texture margin approach?
      • Then smear and blur brush will be a big upcoming task
      • Projection painting brush right now is quite slow
      • Texture painting on low poly objects is very slow
        • PBVH doesn’t get split as much
        • Entire texture is painted at once
        • Needs better approach
        • Divide each PBVH tree into individual texel nodes?
    • We discussed approaches to do the rasterization watertight and more efficiently

  • Sculpt vertex colors patch

    • The state of the patch needs an update
    • Vertex paint refactor was added which lead to some confusion
      • We should unmerge vertex paint changes and commit it as a separate patch first (D14179)
    • The patch needs another round of testing to see if all bugs are fixed
      • Create a built and share it with artists
    • Should be feature complete by now
    • Renaming from ‘vertex colors’ to ‘color attributes’
      • Needs a final stamp of approval and then completed in the patch
    • Go through remaining patch comments and mark them done
  • Add reviewers for D14272

    • This will enable long term refactoring for sculpt mode
  • Make next module meetings more public?

    • Openly invite anyone who’s a developer or interested in contributing in development
    • Share meeting notes and meeting link more publicly as well
    • Users who are interested in the meeting are best to read the meeting notes afterwards
23 Likes

yay news!

(I can’t be the only one looking for meeting minutes)

3 Likes

cool, but why colour attributes? Just Attributes would be better, and then you could choose what your painting (int, bool, vec3(color),vec4(color + alpha),float), and where you want them stored (face, edge, vertex). In int, and float modes, the user could set the min/max value to be assigned to the pressure sensitivity range of 0-1 (like weight paint mode), bool would assign 1 to any pen pressure > 0, and vec3 and vec4 would have colour pickers (like vertex paint).

Quite confusing to have multiple variations of attributes when they should be recognised as one thing; storage of a value in a geometry element (face, vertex, edge, full object). Better to consolidate all to a single painting editor called ‘attribute paint’, rather than having one for colour (vec3), one for weights (floats), non for bool, non for int, and no ability to paint attributes onto any element but vertices.

In the properties panel’s data tab, the value range for each attribute could be changed after the fact, if for example you decide you want your painted 0-1 range to represent -50 to +50 instead of 0 to 10 for example, or even change the datatype from float to bool etc.

1 Like

Great to see progress in texture painting.

I’m sure you guys get a lot of feedback but can’t resist putting in my 2 cents.

Would be great to see tests with multiple UDIMs as well if you’re checking for bottlenecks. A basic character in vfx production can easily have 50-100 UDIMs, some assets even reaching the limit due to naming rexstrictions.

That said, the geometry node module is so gone

just a little artist here, I love the transparency of these meetings. makes you really appreciate and understand a little more, the effort behind these features. Thank you for your hard work!

1 Like

Is there anywhere we, users, can try this prototype Brush?

1 Like

The idea is to keep a distinguishing between old vertex painting workflow and attributes for geometry nodes.

If you keep everything under same list, you need a way to separate layers of your painting and layers of geometry building.
Because that would be annoying for painting workflow to constantly pick a wrong layer that you should not modify or have an hard time to find the desired one in a long list.
That would require a more efficient filtering of attributes list.

No build has been done with buildbot.
Name of branch is temp-3d-texturing-brush-b.

2 Likes

In the properties panel data tab, you could just have a single attributes list to replace vertex groups, vertex colours, uv maps, attributes, face maps.

The list could have two dropdowns; ‘element’ (choose between face, vertex, edge, and full geoemtry), and the second dropdown, ‘attribute type’ (colour3, colour4 (or vec3, vec4), float, int, bool). It would then be very easy to locate your colour attributes, and it would also mean you could do face paint if you’re into running around roaring like a tiger.

The attribute paint editor would replace vertex paint and weight paint, and it would automatically populate with tools specific to what you’ve chosen in the attribute list in the data panel.

Having many lists and editors for the same thing is starting to get really messy, and the lack of unification is also making workflow extremely cumbersome and often limited. Different modifiers only being able to work with vertex groups for example, and no ability to paint anything but colours onto vertices.

1 Like

I believe this duplication you’re seeing is because Blender is transitioning from fixed data layers such as vertex groups or UV layers to generic attributes. There needs to be a period of time during which the two coexist, and that’s now.

Last I checked there were plans for an “attribute edit mode”, but that may have changed, I’m not sure.

In any case the painting module seems to be rethought from the ground up, it’s unusual, I like it. Great times ahead for sure.

Having many lists can have an advantage.

You mentioned vertex groups.
Vertex Groups can be used to store Vertex Mask while painting Weight Groups. That is same attribute. There is no problem.

But when you are painting vertex colors or UVmaps, that is no more same attribute. You have to constantly switch attribute type to select mask or layer.
Same problem would occur if Face Maps were used as Face Masks, instead.
You would have to switch constantly dropdown menu during painting.
For painting, you need, at least, two lists. One for painted layers and one for masks.
Or a list that contains all attributes and a list similar to 2D painting programs that handles painted layers and masks where layers and masks can be attributes of any kind.

And if you want to convert attribute type of a channel to another one ; you will probably prefer to see lists of both types, at same time, to avoid naming problems.

And layered textures could complicate that.

I think layered data should probably just be listed as arrays in the data panel’s ‘attribute list’, using an ‘array’ checkbox to list them. If vec3 were chosen from the ‘attribute type’ dropdown, and you then checked the ‘array’ option, it would list all vec3(colour) arrays (aka multi layer) for the chosen element type. In the ‘attribute paint’ editor there would then be a second list which showed all of the layers from which you could select quickly to view or edit the data. Additionally, a second list could dynamically appear below the ‘attribute list’ when the array checkbox is checked, so that you could still have visibility when not in the ‘attribute paint’ editor. Although perhaps the spreadsheet should completely replace the lists in the data panel.

I think your right about it being potentially cumbersome to switch back and forth for masks during painting, so maybe a good solution would be to be able to mark certain attributes as masks, and then have a second list in the ‘attribute paint’ editor itself which would show them for quick access. I can’t see the need to paint uv’s, but if you could also mark attributes as ‘uv’ so that they would be included in places that list available uv’s, such as the shader editor’s uv node. Perhaps there could be a dynamic checkbox on the ‘attribute list’ which dynamically altered how the data is displayed dependant on the active editor. For example in texture paint mode, it could switch to only show attributes marked as uv, or in sculpt mode ‘masks’, etc.

All modifiers could replace the stringent vertex group properties with attribute properties, allowing any modifier to use any attribute that has a compatible data type (decided by the developer). Then we could use dynamic paint for example to automatically populate an attribute, and then use it anywhere. At the moment you have to run scripts to convert between vertex paint and vertex weight etc, which takes away the dynamic element.

Currently Blender’s extremely restrictive in terms of possibilities because of the existing differentiation between what are effectively different flavours of the same thing. Unification will open up a level of power that’s currently unachievable.

That is a huge amount of UDIM’s is there something you could share so we could test how it currently performs? And where we should improve

It’s quite common in VFX. I worked on several assets in the 500+ range, but even an average human topology which usually gets re-used even for mid/background characters has 50-100 UDIMs (typically since rigs/textures and whatnot are all transferred between shows for generic characters these don’t change too frequently and need to be future proof).

Add to that I worked on productions where they bumped up the resolution on those to 16k with the possibility of going 32k. (this would be usually extreme cases)

It seemed pretty excessive to me but I’m working in the assets realm so I’m sure people making those decisions knew what they were doing.

If it’s helpful for you guys I can dig around for a generic topology I got at home and try to reproduce a similar UV layout for let’s say a human character.

2 Likes

If that is possible that would be extremely helpful. The best man to setup such a model would be someone who usual is working with those. We should otherwise make to much of assumptions and might not reflect the intend.

Most important would be the number of objects/vertices/faces/uv maps/textures/udim tiles/dimensions.
We might not be able to support it from the start, but would be really helpful to get the initial design right and as a stress test. The texture doesn’t have to be painted as that is the goal of texture painting.

Do you have contacts with people who make those decisions. Would really like to understand the reasoning behind those decisions. I see a lot of issues handing 32k textures as they cannot be handled by GPU hardware natively without a lot of trickery.

1 Like

Definitely. These decisions are never taken lightly and are not very frequent. It’s the cases where for example late in the show the client changes a camera for a shot, and what suddenly was far away is now right in front of the lens. Add to that an asset too far down the pipeline with all sorts of dependencies, and sometimes the least harmful (but still very painful) way is to up the res.

I just mention it because it does happen from time to time, and if it’s not possible one would have to re-create a sometimes complex setup/nodetree in a different package. It’s always good to know the limitations.

I sent you a PM with a generic character topology that has a very similar UV layout than those used in production. It does hold up in a wide variety of situations and it seems that the number of UDIMs isn’t a problem for other departments either. We always need to keep in mind for these assets that they need to be future proof to some extent. With more and more movies going 4k it does make sense I think.

If you’re looking for extremes most often the bigger spaceships or Pacific Rim like robots are a good candidate for high UDIM count.

“The number of textures needed to cover an asset was massive, up to 1000 UDIMs per Jaeger.”

I can ask about people from other departments closer to the rendering/lighting end since they have a better understanding of these limitations, and come back to you with some contacts if it’s useful.

As for decision makers, I tried to approach some higher up co-workers a couple years ago and all of them were pretty reluctant to get involved. Conflict of interest, NDAs and Blenders GPL license and open-source nature was among the main reasons.

3 Likes