Paint Mode: Design Discussion & Feedback

Since Texturing became one of the main development goals for 2022 there have been a few improvements in progress.

An important missing part of that design is still the Paint Mode itself. This post will explain the current conclusions and open questions of where we are heading for the most important functionality.
By sharing this we hope to get more feedback and suggestions from the community.

Some of these features would benefit greatly from using the proposed texture layer nodes.
But until then we will make the features usable with the currently available features.

Based on Sculpt Mode

To make the mode structure clearer we are aiming to use Pablo Dobarroā€™s design for modes.

Paint Mode will be based on the code of Sculpt Mode, which means it will inherit a lot of its features and fast performance.

The mode will replace the existing Texture Paint and Vertex Paint Modes and will provide the full tool set for artistic painting of color attributes & image textures.
What you will paint on can be set via the Canvas selector.

Toolset

To cover all workflows the Tools need to be extensive. A few tools and general operators will be the same as Sculpt Mode, like masking, face set and automasking features.

But overall the Toolbar would include these most essential tools:

  • Paint Brush
    (A new 3D painting brush based on the new sculpt mode paint brush and the work on PBVH texture painting.)
  • Draw Brush
    (A simpler brush meant for pixel art. Will make it possible to paint single pixels and faces.)
  • Projection Brush
    (A brush that is much closer to the original texture paint brush. This one will use proper occlusion for 2D projection painting and use of stencil masking.)
  • Erase Brush
  • Smear Brush
  • Blur Brush
  • Clone Brush
  • Draw Face Set
    (Used for drawing face sets. More on that further down.)
  • Mask Brush
  • Sample
  • Mask by Color
  • Color Filter
  • Gradient
  • Gesture Tools

Masking Modes

In the current painting modes there are various ways of masking. Including all of these in their current form in a single Paint Mode would lead to a muddled and confusing design.

But still, these ways of masking should be supported:

  • Masking per vertex (Like Sculpt Mode masking)
  • Masking per face corner (To achieve sharp edges on color attributes)
  • Masking based on selection (Synched with edit mode)
  • Masking based on a texture (Stencil Mask)


(Mockup of masking modes)

The proposal is to add three easily accessible ā€œMasking modesā€ that can be switched with the number keys. This directly parallels the selection modes in edit mode, grease pencil and the UV editor. And just like in these other modes they should be interchangeably usable for intuitive and fast masking.

The 3 modes would let you:

  • Mask Vertices
    Same behaviour as sculpt mode.

  • Mask Faces
    Also similar to sculpt mode but with the ability to mask face by targeting per face corner data.

  • Mask Pixels
    This feature should replace the current ā€œStencil Maskā€ but with full support for sculpt mode masking features

These masking features are meant for loose and temporary masks, similar to selections in other modes. For more permanent masking for material and texture mixing it would still be referred to paint image textures and use them as masks in the material.

Open questions:

  • Will the sculpt/paint mode mask need to be always stored as face corner data to make this possible?
  • Available masking modes likely depend on what data is being painted? (Vertex colors, face corner colors, image texture)
  • How can, or should, image based and geometry base masks be synced? Syncing them would make switching between them more intuitive, but only if this doesnā€™t result in data loss for masks
    Perhaps Pixel masks need to be an opt-in feature. If disabled the masking features will target actual vertices and faces. Once enabled all masking modes and tools will result in editing the image texture of the pixel mask.
  • Pixel masks will likely be stored in the object for now. But should this eventually become part of the Texture Layer design? This would make it easier to keep the resolution of the painted textures and pixel mask in sync.
  • For pixel masks do we need better automatic UV map creation? The use and creation of pixel masks should be as easy as possible.
    Until then it might be necessary for users to create/define a texture and UV map just like in the current stencil mask options.

Syncing Edit Mode Selection

A very useful feature of the current painting modes is that the mask and selection are synced. This way masks can be much more precisely created in Edit Mode.
While switching between Edit Mode should not be required - masking in Paint Mode should be intuitive enough - some workflows rely on this for painting precise attributes and technical textures.

So keeping this functionality will be a priority!

Supporting this will not be straightforward. Paint Mode masks can either be per face-corner floating points or RGB image textures, while Edit Mode selections are boolean values.

In the case of per-face corner floating points there can be a threshold of what value is considered selected in Edit Mode. The ideal value would probably be 1. So any vertex that is completely masked will show up as selected in Edit Mode.

Itā€™s important that switching modes will not result in data loss for masks. So only if any part of the mesh is being deselected or selected, will those elements be updated in the Paint Mode mask. The behaviour should be predictable and intuitive.

Open questions:

  • How can, or should, the Edit Mode selection be synced when using the pixel mask?
    If no clear solution can be found we could disable syncing with selections if pixel masks are used. This would relate the the previous open question of making pixel masks an opt-in feature

Inverted Masking

The typical workflow issue when using masks is that users mask a section of their mesh and then invert the mask.
In most cases the areas you mask are the ones you DO want to edit.

So a long time proposed change can finally be implemented by inverting the behaviour of masks in general. This would also make it much easier to sync selections and masks, since they both define the elements you CAN edit.

The expand operator has already used this updated design ever since it was released and it led to a more usable tool.

Open Questions:

  • The name ā€œMaskā€ might end up confusing if the behaviour is flipped. You are no longer masking areas then, you are masking everything else.
    Renaming them to ā€œSelectionā€ might be a good change, especially if masks are synced with Edit Mode selections.

Mask Overlay Color

The mask overlay should also be updated to make it more useful for painting workflows.


(By adding a color option, it should be easier to customise the look of masks for texturing needs. Credit for the model shown goes to Daniel Bystedt)

Instead of only having a slider for multiplying black on top masked areas, there should be options to mix in a custom color.

Saving Masks

A small addition that is much requested and would end up very useful is to save and load masks for later use.
This is useful when an intricately created mask needs to be changed or discarded but would result in too much time lost.

This would be as simple as saving geometry based masks as generic ā€œFace Corner - Floatā€ attributes. Since pixel based masks would require an image texture it is already easy to replace and store them as well.

ID Maps and Face Sets

An important feature that will be necessary to implement with the upcoming Texture Layer design are ID maps. They are already very similar in their use to Face Sets and can help in effortlessly creating masks and layering materials.

There are issues with storing and using ID maps as single images though:

  • Anti-aliasing between id colors usually result in visible gaps. The alternative is not using anti-aliasing which is also not ideal.
  • Too similar colors can result in bleeding and inaccurate masks
  • Compression can lead to inaccurate masks

The proposal would be to instead create a unique version of this concept in the form of ā€œPixel Setsā€ within Blender. Instead of saving a single image, we could save a set of grayscale masks that are auto-normalised to avoid overlaps between them.
This way we can avoid all issues mentioned above.

Just like with Masking Modes, there should be a toggle to use either Face Sets or Pixel Sets, if available. If Pixel Sets are used they would offer the same features as Face Sets for quick control of viewport visibility, creating masks and automasking.

Open questions:

  • How can ID maps be imported and exported while Blender uses Pixel Sets internally?
  • Do Pixel Sets have to be an opt-in feature like Pixel Masks?
  • Pixel Masks and Sets should ideally be part of the Texture Layer nodes to keep them consistent in resolution and use UV maps with the material that is benign painted.
    Are Pixel Sets until then even useful or should Face Sets be enough?
  • The use cases for using ID maps for the same functionality of Face Sets is not clear. Would this be useful or would Face Sets be enough for managing temporary masks and viewport visibility?
    If so, then it would be better to keep using the term ā€œID mapsā€ and only use them for easy material layering.

Dynamic Overlays

Just like mentioned before with the mask overlays, itā€™s not ideal to have the color information on the object obscured.

This was already a long term concern with Face Sets and should finally be addressed with Paint Mode.

The issue is that the Face Sets - and eventually also Pixel Sets - are always visible which makes painting hard or they are completely turned off which makes it hard to use them.


(Current way of drawing face sets and proposed contours at 100% opacity)

The proposal is to add an option to make the overlay dynamically visible whenever it is used, edited or otherwise necessary to see it.

For example when using tools like Face Set Draw or Expand while snapping to face sets, you would see the face sets completely filled. But otherwise face sets would be hidden.
If options like automasking by face sets are enabled it would be best to show the face sets in a more subtle way like contour lines, so that they are less distracting.

Image Editor

While updates to the Image Editor are also important, this is its own big project that should come after Paint Mode. But to make the feature set complete itā€™s important that the same or similar overlays, tools, operators and masking options are in sync with the image editor.

This can also be explored further with the inclusion of the Texture Layers design.

Feedback wanted

These are a lot of design decisions that will have an impact on Painting and Sculpting in general. To make this as intuitive as possible we want to get more feedback from as many different perspectives as possible. Nobody should be left out.

If anything in the proposal wasnā€™t clear enough also donā€™t hesitate to ask questions.


Important notes from Feedback so far

  • Inverting the masking behaviour everywhere for everyone should also be a user preference or tool setting. That way it should still be possible to create a mask brush to mask off small areas by just painting with the brush. Same for gesture tools like lasso masking.
  • More good reasons to rename ā€œMaskā€ to ā€œSelectionā€ in sculpt mode and paint mode is to make it obvious that they are meant to be quickly created and discarded, like edit mode selections.
    If users want to create a proper mask, they should create an attribute or image texture to paint on, which can then be used for mixing/layering in shader and geometry nodes.
    These are two very different use cases for masks, so naming them something different makes sense.
  • The dynamic overlays could also be extended to masks. So if no masking tool/brush is used, the mask would be displayed as a contour line
  • Greying out or even hiding selection modes depending on the data you are painting on would not be a good idea. Especially in Paint Mode, the user could switch between painting on various color attributes and image texture regularly. Selections/Masks should stay consistent and understandable when doing so. Same when switching to Sculpt Mode.
  • Converting the painted mask whenever the mask mode is changed, or when switching to Edit Mode would very likely be the easiest and most usable solution. Trying to propagate or abstract the mask between modes would make the behaviour far less predictable and for questionable advantages. Selections are meant to be temporary and quickly & regularly discarded after all.
55 Likes

Whatever you do, please include design that supports floating point brushes in Image Editor or whatever it takes to support pixel perfect editing in Image Editor. Itā€™s been almost 3 years(I think) since the last movement on this when Pablo Dobarro retracted his floating point brush size patch because of conflicts with the rest of the brush system.

6 Likes

Yes. I mentioned this tool as the ā€œDraw Brushā€ and it would be good to include this feature in Paint Mode from the start :+1:

8 Likes

I like the design (keeping in mind that some of it might change with the layer system arriving of course)

  • Regarding the edit mode syncing of masks - would that make painting masks on an object with 1-2M polys tediously slow (considering making selections on objects with high polycounts is not exactly fast) ? Using mid poly meshes for texture painting is not uncommon, and for environments or more complex assets it might even go higher. While it seems like a nice feature I donā€™t think itā€™s as essential as having fast painting performance. Iā€™d prefer keeping it optional in any case.
  • Not just for masks. Zbrush and Substance have pretty solid auto UV options, using them in conjunction with face sets can be a huge time saver. Iā€™d say itā€™s essential to have better auto UVs for any texturing workflow nowadays.
  • Please donā€™t. Or make it optional and preferably not the default. Itā€™d be very confusing. Having the masking marking menu from sculpt mode to quickly invert, blur, sharpen and clear masks is already fast enough to tackle this issue in my opinion.

  • Since I see no mention of symmetry Iā€™d like to add that to the list. While I know this is not RCS Iā€™d still like to see topological symmetry which would be very handy for both sculpting and texture painting.

  • Painting in UV view is definitely a must have though. Currently thereā€™s no way of displaying UVs in object mode meaning no way of knowing where Iā€™d paint on the object. (Imagine working on something like a house or a prop)

Generally the proposal seems to be focusing around characters - adding other use cases like large scale environments, complex props or vehicles to the equation might be useful.

Thank you for keeping this an open conversation.

10 Likes

(Iā€™m trying to keep this question separate since a bigger part of the proposal seems to resolve around this design)

Why though (genuine question) ? While re-using masks is great, I donā€™t see the edit mode conversion of masks as a handy feature, or something that Iā€™d ever use to be honest (let alone all the time for every mask I paint). Modeling mode is for creating and modifying geometry, I donā€™t think that heaps of painted and (once the layer system is there) generated maps existing as edit mode selections would be something that many people would use.

I worked in Mari and Substance for ages in production and while often felt that having edit mode selections available for masking purposes is extremely handy, I never felt the need to have it the other way.

1 Like

In expand operator, there is ability to invert its default behaviour.

In fact, choice depends on ratio of what is modified.
Currently, you want to preserve a small area and edit the most part of object. You mask small area.
You want to edit small area and preserve larger one. You fill object with mask and then, unmask small area.
Inverting default behaviour should not prevent user to clear global mask and just mask a small area.
In any case, it is more efficient to paint smallest area.

I think that automatic mask filling of whole object or not probably should be a user preference or a mode option.

I donā€™t have strong opinions about open questions. I am satisfied by design. It works for me.

1 Like

Regarding the edit mode syncing of masks - would that make painting masks on an object with 1-2M polys tediously slow

I wouldnā€™t worry about this yet. Thereā€™s no way to say that this will definitely be an issue.

Please donā€™t. Or make it optional and preferably not the default.

I donā€™t think thereā€™s a way to make such a feature change optional. In the long term I think this will be the right decision. We can already see in the current painting modes that it works well. And it will only be done for sculpt and paint mode if itā€™s just as or more intuitive to work with as the current masking behaviour.

Iā€™d still like to see topological symmetry

That would be an interesting feature for most modes indeed! Not essential for the release of Paint Mode though.

Currently thereā€™s no way of displaying UVs in object mode

Thatā€™s a good point for the image editor!

adding other use cases like large scale environments, complex props or vehicles to the equation might be useful

If you have suggestions or requirements, feel free to share them.

While re-using masks is great, I donā€™t see the edit mode conversion of masks as a handy feature

Since Paint Mode can also be the mode for technical painting of textures and color attributes for external game engine and for geometry nodes it would be very useful to keep masks and selections in sync.
Some artists rely on being able to make precise selections in edit mode instead of using brushes with a falloff.
There are also moments where you want to reuse your current sculpt mode mask in edit mode and the most common workaround so far was to use the viewport visibility of the mesh (Which is already being shared across modes for better usability).
So at leas to masks on the geometry itself I see this as a good inclusion. For texture images itā€™s still questionable.

1 Like

I think that automatic mask filling of whole object or not probably should be a user preference or a mode option.

Maybe something wasnā€™t that clear with the proposal to invert the masking behaviour of Sculpt Mode & Paint Mode.
It doesnā€™t mean that using a Mask Brush will mask everything else around your stroke. It means the areas you masked are the areas you can edit.
Thatā€™s why changing the name to ā€œSelectionā€ would make this less confusing. The areas that are selected with a ā€œSelection Brushā€ or ā€œLasso Selectā€ would be the areas you can paint on. It communicates more intuitively what the inverted actually behaviour means.

3 Likes

Thank you for answering my questions.

Out of curiosity - what is that based on? Iā€™d be interested to see the results of a widespread poll. Preferably with people who used a wide variety of relevant softwares over the years.

Essentially what youā€™re saying here is that every other software does it wrong, and Blender will do it the other way. Thereā€™s been a history of doing things like this and it didnā€™t always turned out great.

With all due respect, you need to keep in mind that every artist has different workflows and preferences. All Iā€™d wish for is that decisions like these are made based on the consent from a wide as possible audience.

I mean this is not trivial at all - if I got it right, masking things would mean masking everything else under the hood(?) What if that would be a different tool called unmask, and we keep masking to do what it does?

edit - Scratch that. Naming it selection (and keeping mask brush?) and weā€™re golden I think.

What made me bring this up is seeing Suzanne subdivided twice and a lowpoly head in the design document. It would be reassuring to see use cases which would stretch and test the boundaries of a system in order to make sure itā€™s functioning properly. Not saying that it doesnā€™t of course.

Nothing specific apart from using something like a detailed car model, a very large spaceship, a block of furnished apartments or a square kilometer of rocky terrain in addition to character model, when working around a design. Iā€™m sure it would bring up some very different issues.

Please donā€™t get me wrong, Iā€™m not trying to be difficult, but I donā€™t see use cases where transferring masks to edit mode are useful. I agree textures and color attributes are a super important thing to have, but I donā€™t see the specific correlation with masks. I agree precise edit mode selections is important and Iā€™m all for brush based selections. I donā€™t think that I would go to paint mode masking in order to make one though.

This would be something Iā€™d be interested in knowing more about. Wouldnā€™t it be more handy to have face sets quickly available in edit mode? I donā€™t see mask syncing being that crucial or even super handy - would be interested to see what others think.

edit - Maybe scratch the last two parts as well. If itā€™s the ability to make selections in texture/color paint mode and transfer those to edit mode I donā€™t have a strong opinion about that. Might be handy in a non linear pipeline.

Itā€™s great to see some action in this area. Looking forward to see where things end up.

1 Like

OK. I had difficulties to integrate that mask will be stored as a named item in a list.
So, probably, there will be settings to fill whole object or some face sets, at mask creation.

But semantic changes nothing to my remark.
There is no problem in renaming mask/unmask by preserve/select.

What I mean is that those tools should continue to allow user to invert behaviour while using Ctrl key.
And ideally, default behaviour of those tools should be a user preference.

Gradients in masks are a must-have.
One feature that users are missing from sculpt-dev branch are IP masks to control mask boundary through A pie-menu.
This and Mask Select tool that is an active tool version of expand tool would be welcomed in a Paint mode.

1 Like

Iā€™d be interested to see the results of a widespread poll. Preferably with people who used a wide variety of relevant softwares over the years.

Hopefully with this thread itā€™s easy to show, share and discuss these and have more viewpoints heard. The pool of voices these conclusions are based are still small.

Essentially what youā€™re saying here is that every other software does it wrong, and Blender will do it the other way.

Other software packages are doing temporary masking in various different ways but they can be simplified into two camps:

  • Painting masks. Usually via floating points or pixel values. Mask = Canā€™t edit
  • Creating selections. Usually via loose meshes, UV islands and faces. Selection = Can edit

Since both these methods were highly requested for Paint Mode the idea is to try and make them a single unified concept under the hood.
I think your suggestion to still include the ā€œMask brushā€ as it behaves right now could work. That should be considered for the other masking tools as well.

3 Likes

Thanks for explaining, things are a bit clearer now.

Might be hard to unify those two things though - per pixel masks require UVs and can have a range of values from 0 to 1 where as subobject type selections would require consistent vertex IDs and are usually true/false.

One object could have UVs and no multires while the other one no UVs and multires. I guess the whole thing would need to connect with nodes somehow down the line as well. If there is a way to marry the 2 ways of working I canā€™t see a problem with it if it doesnā€™t have a negative impact on performance.

Iā€™d almost gravitate towards saying that in terms of texture painting masks are so different from selections that Iā€™d just make it a different tool and save myself the time. Might as well create that mask extract tool which might be about the main thing anyone would use masks as selections for and wait for more use cases to justify a larger effort?

Then again, if thereā€™s a time to do big things itā€™d be now when a system is re-built.

Iā€™m sure there are some time and budget constraints on the project which is the main reason I tend to choose the more straight forward solutions. Main thing Iā€™d be looking for is a reliable system, a nice UX (with easy and fast switching between modes and less of those undo glitches) and maybe some procedural methods to save me some time.

2 Likes

A small question: Will this Paint Mode also support painting attributes like Roughness directly on objects?

Edit: I am taking @JulienKasparā€™s heart as a ā€˜Yesā€™ response to my question.

1 Like

Just want to add that regardless of how the new mode will work. the external quick edit button is just TOO useful and doesnā€™t deserve to be so hidden from sight. its also too useful to be accidentally removed during the transition so please keep it atound.

image

8 Likes

Looks promising!
I see this toolset as useful and diverse and that it will fulfill my needs.

For the masks, I am very used to texturing softwares where the mask is connected to a texture layer rather than storing it in a separate list. I am curious to see how the Masking Modes, the Saving Masks feature, and the Texture Layersā€™ mask will be able to correspond to this workflow.

The idea of inverting and renaming ā€œmaskā€ to ā€œselectionā€ could be useful for new artists but confusing when transitioning from other tools. An alternative is to call the temporary masks as ā€œselectionsā€ while saving a selection would be called ā€œsave as maskā€ so that ā€œmasksā€ are used for more permanent use.

so that ā€œmasksā€ are used for more permanent use.

Thatā€™s one of the key motivations for this. ā€œSelectingā€ areas in edit/sculpt/paint mode will then be something different than creating ā€œMasksā€ for repeated uses in shader/geometry nodes and layered textures.
Selections are always meant to be fast, easier and temporary :+1:

2 Likes

really like the idea of ā€œā€œselectionā€ā€ instead of masking, its anoying to mask the area that i want to work and then need to invert the mask, the concept of masking its good there are times that an ā€œinverted selectionā€ its good but most of the time im sculpting i end up using the invert mask to work the area i have masked.

2 Likes

Absolutely. For sculpting many people use the mask/invert workflow at times.

On the other hand, itā€™s probably not exclusively that workflow though - I payed more attention to how I was working throughout the day and found myself masking out fingers, eye areas, lips and parts I wanted to keep untouched instead of masking out the whole body/head except those parts. In other instances I masked out an arm/leg and inverted the mask. It very much depends on the situation.

We need to keep in mind though that masks are something entirely different for texture artists. Depending on peoples workflow and personal preferences they might fully mask out a layer/mask and paint in the values or create an empty mask and paint masks wherever needed.

Since masks are such a widely used concept throughout both 2D and 3D applications Iā€™d argue that their behavior is best left consistent.

Not saying there canā€™t be other tools to do the exact opposite, but I probably wouldnā€™t call them something thatā€™s commonly doing something else. Like having a select tool that deselects cause of reasons.

As for pixel sets/ID masks and face sets - My vote would be for just keeping face sets for now until the definite need for something pixel based arises. I think hiding pre-defined parts of the geometry on a subobject level should be sufficient for most people. Unless I got it wrong (pixel sets essentially being face sets stored as grayscale masks)

5 Likes

Iā€™m slow and I need my time, but the more I read the proposal the more I like it.

I like the idea of synchronizing the selection even if itā€™s lost when going into Edit Mode, and swapping the mask for selections. Anyone who wants to keep the current workflow should be able to do it, but a priori it seems more consistent, even if itā€™s not standard.

It is true that in sculpture it is common to invert the mask, and in 2D painting programs it is common to select the area where you want to do the intervention, so for me it makes perfect sense to invert the process.

Congratulations.

Just my two cents ā€¦ sorry if itā€™s redundant.
Paint should not be a mode, it should be an option in sculpt.
Paint option should not be rgb but bsrdf/displacement with a nodegraph based brush.
I hope It make sense to you.