Trying to implement polygroups [planning stage part 1]

Feature :
Sculpt layers will be arbitrary selections of verts that are stored.
Users may mask/unmask these sculpt layers, for the purpose of working with large complicated meshes more easily.
Sculpt layers must survive/work with dynotopo.

Data :

  • Create a GHash where the key is a vert pointer and the value a list of layer indexes that belong to this vert.
  • Add a list of layers to SculptSession.
    Each layer stores pointers to BMVerts.
    To mask a layer iterate over the layer and use BM_ELEM_CD_SET_FLOAT to set desired mask value for the vert. (its faster that iterating over the entire mesh)

Modify Dynotopo so that:

  • vert deletion causes removal from GHash and layers.
  • vert creation causes addition to the GHash and layers.
    Decide to which layer to add by checking proximity to other vert layers.

Persistence :

  • when entering sculpt mode delete all layer data and restore it from CustomData.
  • when exiting sculpt mode delete CustomData and rewrite it to match current layer data.

To be continued in part 2 (the focus of further research and investigation)
Write to disk, Expose to UI and Create the UI

The question is:

Is the plan any good ?


The main complexity here is dealing with the different mesh representations, there are regular meshes, multires meshes and dyntopo bmeshes. The proposal mostly deals with the dynamic topology bmesh case I think, it’s not entirely clear to me how it extends to multires for example. It’s reasonable to ignore that for now though and make one case work well first.

Is having a layer exist for only part of the vertices really important, is that how other software works? I was expecting sculpt layers to exist for all vertices. That would avoid the hashing, which I expect to be rather slow and memory intensive.

The most similar thing we have to sculpt currently is shape keys, and to edit those we basically replace the vertex coordinates with the shape key on entering edit mode, and then sync things back on exit or when switching active shape key. This means the transform tools can be unaware of shape keys, and topology changing tools must only preserve per vertex customdata.

I wonder if you plan to handle sculpt layers similarly, i.e. the sculpting tools would just move the vertex positions unaware of the layers, and then it would compute the delta afterwards? Dynamic topology operations could also treat sculpt layers as generic customdata perhaps, which would then also allow preserving data like vertex colors?

I have yet to consider the other mesh representations due to the fact that advanced masking operations (and thus layer operations) require linking data for stuff like growing mask selection, fill mask hole or mask linked (very useful).

As for the sculpt layers they are nothing more than selections of vertices.
Instead of manually masking/hiding a portion of geometry each time you what to work with that portion of geometry only, you would just mask/hide the corresponding layer.

In Zbrush the way it works is that each vertex/face can only belong to one polygroup, thus each vert would simply store a int that corresponds to a layer number.
By default all vertices of a mesh belong to an initial default layer.
This is relatively straight forward to implement, but has limitations in the sense that it’s impossible to have “categories” or “overlaps”.
Eg. a polygroup can’t contains vertices from other polygroups
Eg. you cannot share vertices between polygroups (except only at boundries)

Ok, then I wouldn’t call this sculpt layers since in ZBrush or Mudbox that has a very different meaning.

If this is like polygroups, then I wonder if the extra flexibility of allowing polygons to be in multiple groups is actually useful enough to be worth the extra complexity for both users and the code. Is there a specific reason this extra flexibility is needed?

This is from Zbrush:

Lets say I want to select only polygroups of the head, in Zbrush the workflow is :

  1. use rectangle/lasso brush to hide the majority of the head vertices in one move, this leaves pieces of polygroups
  2. shift click on each remaining pieces of polygroups to cleanly hide the head portion

Now that think about it more, there doesn’t seem to much need share vertices among these layer that much.
In the sense that nobody has these demands for big hierarchies of layers, I could get away with making the verts be shared by at most 3 layers.

I’m not sure if having multiple layers per object (or should we name it polygroups to keep it in zbrush terms) is really needed. As there are already the objects to separate things (or subtools) from each other. Sure it wouldn’t hurt to have it, but it might add unwanted complexity using such a feature? Don’t know nothing about the implementation, but looking at zbrushs way of handling the UX on this is rather lightweight. Making all the tools (selecting/hiding/assigning polygroups) to work great even with a single layer sounds like quite some work too…

Maybe more non-destructive workflow.

And can we also get painting layers in Texture mode?

So I guess really what is need is some kind of hierarchy of polygroups, rather than arbitrary overlapping ones? Each polygon could store one polygroup index, and a polygroup could store a parent polygroup index. This would give minimal memory usage, and Blender selection tools could be made aware of this so you can easily decide which level to select at.

Eventually after thinking about bone weights, I decided that having polygons be shared by multiple poly groups is a bad idea, as sorting between them is tedious.
Instead have 3 layers of hierarchy :

  • first level works exactly like Zbrush polygroups, a polygroup me not be shared among other polygroups
  • above first level polygroups will exclusively be comprised of lower level polygroups

When is the support for multires in sculpt mode coming back ?

It’s being worked on right now, so likely in the next two weeks or so.

1 Like

Can the title be changed to properly reflect the topic? This is not about sculpting layers at all, is about polygroups from what I read.

I’ve seen people in other sites (blenderartists, rightclickselect, facebook, twitter) talking about sculpting layers being implemented in Blender and linking to this thread.
Thanks :slight_smile: