Subsurf Modifier preserve custom normals

Archive Blender Development
1

I have started to work on trying to add an option to the subsurf modifier that will preserve the existing custom normals of a mesh and interpolate the loops normals to the newly generated geometry.

In NPR stylised modeling custom normals are extremely important in getting nice shading on your characters. Currently the subsurf modifier overwrites the existing loop normals.

I spent some time testing and breaking the modifier to try and understand how it works. I currently have it working to where I can give the resulting mesh custom normals. Right now it is just a simple vector for each loop not actually based on the geometry.

In the subsurf code there is a CustomData_interp function that interpolates the loops UVs between the source mesh and the resulting mesh. In that function it also can access the custom normals of both meshes, but the way it is setup currently it skips this when it calls a if (!typeInfo->interp) {} check.

I tried to manually force it to skip this check for the custom normals layers, but it resulted in crashes. Not sure why it crashed, but I will keep prodding at it to see if I can get it to not crash. I just want to see if the interp function could work with the custom normals layer as it is.

EDIT: seems it crashes because the interp check is not just a boolean check or something similar like I thought. The typeinfo->interp is a function itself. So by bypassing that it was trying to interpolate the data with no actual interpolate function. So it seems like that may be where I need to modify the code to interpolate the custom normals.

I just wanted to share this to see if anyone else maybe has attempted this or has some ideas of how to approach it.

4 Likes
2

I have got it partially working. It works for quads, but not tris and ngons atm. This is because I was mostly focusing on quads in testing and may have missed adding to the non quad parts of the code.

I found that it was not trying to get an interp between CD_CUSTOMLOOPNORMAL but getting the CD_NORMAL to interp. I am unsure of what CD_CUSTOMLOOPNORMAL actually does as I have only seen it used to check if there are custom normals. I will have to investigate more to find if it is ever used for another purpose.

Once I added a check for CD_NORMAL existing in the result mesh and if not add that layer to the result mesh then it properly interpolated the custom normals for the quad geometry. Then once all the calculation is done I used BKE_mesh_normals_loop_custom_set to set the data to the mesh.

Top set its no custom normals unsubdivided and subdivided. Bottom set is custom normals unsubdivided and subdivided.

Capture
Capture771×807 20.3 KB

2 Likes
3

I have figured out how to get it to work. There was a flag on the CD_NORMAL custom layer that prevented it from being copied.

For tris and ngons the loop_data is done differently and it copies the data from the coarse_mesh->loop_data. This is different from quads and the flag prevents the normals from being copied which results in no normals for tris and ngons.

I got it to work by just forcing it to copy the normals. I will need to come up with a more elegant solution for it to pass review but for now it is working.

Normals
Normals773×789 62.5 KB

4 Likes
4

Have you tried to create a pull request to add this feature to blender?

5

This feature is in blender. It is called Use Custom Normals now.

2 Likes