Hey everyone,
I ran into some issues with custom normals and shape keys recently. When you create a shape key on a mesh with custom normals you have no control over the custom normals for the shape key only for the base mesh.
I think this is something that I would potentially like to tackle adding to blender, but it is a much larger project than I have previously tackled.
So I thought I would get some peoples input. Is there something preventing this from being an option?
The feature would be great to have, and there is nothing strictly preventing it to be added.
But shapekeys are a rather fuzzy and annoying piece of code currently in Blender, so not sure I would advise adding new stuff there… I’d rather see it properly sanitize first tbh.
Besides that, am not sure off hands how to deal with the cumulative nature of shape keys, normals are not mere offsets like vertex coordinates, this would require some design and research I think.
I would advise against adding this feature in the near future, I don’t think it’s important enough to justify the added code complexity.
That is a good point. Are there any plans to fix up shape keys in the future?
I was thinking of the normals just being a linear interpolation from 0-1, and it would average the various shape keys normals. I didn’t think about the fact that shape keys can go into negatives and beyond a value of 1.
I think an alternative that would fit my and others needs would be to add an option for a shape key not to affect the custom normals. I feel this would be less complex to add.
I imagined something that big would be complex to add in. I think I will try to implement the non affecting custom normals option and put it into a custom build for others to test before trying to get it into master right away.
I find that shapekeys do exactly what I want them to do to my normals, including my custom and per-face normals, so I’m not sure exactly what your goal is, but there are a couple of tools that you could probably use to do it.
I like to make custom normals out of mesh normals and a data transfer modifier, and there’s nothing wrong with keeping that live, and possibly linking shapekeys via drivers between source and destination.
I also find that I much prefer messing with my normals in nodes, and that it’s almost always possible (just not if I want to use those custom normals in the modifier stack, and modifier stack custom normal support is a little iffy anyways.) Worst case scenario, you can bake a per-shapekey normal map and use that, but depending on what you’re doing, some unbaked math might do the trick instead.
My goal is to allow these anime NPR custom normals to be kept consistent when using shape keys to create expressions. The normals are setup to give a very specific shading results and the deformation changes to the normals kills it.
Here is an example. The top model is the hand edited normals and the bottom is how they fall apart once a shape key is applied.
I myself would actually use data transfer modifiers to keep things procedural and have more control. Also since I don’t use shape keys I rely more on bone rigs the armature deformations would also kill the custom normals.
But many NPR users use shape keys for their face rig so I want the custom normals not to fall apart.
I guess I’d need to see exactly what the philosophy behind the normals was (hopefully, not hand tuned to a particular lighting/camera setup, but based on some general sense of how you want them to behave.) A file, which would let me move a light around and see exactly what your normals were, would probably let me get that sense.
But what it looks like from the picture you’ve shown is that your philosophy is mostly flatten your normals onto two planes. That should be working okay with shapekeys, and I’d suspect the issue is less with the normals and more with the topology. (Look at the wireframe mouth. Now google “3d facial topology” and compare with the images you find.) In other words, I don’t believe those normals would look good even if you weren’t using custom normals.
However, you certainly could be grabbing custom normals after the evaluation of the shapekey. If I’m correct about your philosophy regarding your custom normals, one way to do this would be to data transfer (probably partially) custom normals from a copy of your mesh that had been flattened onto two planes. (One way to make this flattened mesh would be a cast modifier, cube-type.)
That philosophy could also be handled in material nodes as well, probably mixing object space normals with normalize(1,1,0) or (-1, 1, 0) on the basis of side (ie, object coords.x > 0.) Normalize after mixing, convert back to world space.
That’s no armature; an armature would complicate things a bit for the node option (for the data transfer, you’d just data transfer from an armature deformed pair of planes.)
The philosophy behind these normals and the topology come from this presentation https://www.youtube.com/watch?v=yhGjCzxJV3E and this pdf http://www.ggxrd.com/Motomura_Junya_GuiltyGearXrd.pdf
This is not my model nor how I would approach creating my characters btw. This is just a very commonly sought after type of character workflow for 3d anime. It is more than just flattening to 2 planes but that is apart of it.
I am aware of normal facial topology, but this style of modeling is not concerned with it and doesn’t need it with proper custom normals control.
I do use the data transfer modifier in the way you describe. I use rigged planes to control the direction and vertex groups to control what areas it affects, but others do not. They want to be able to edit the normals and have it stay consistent when using shape keys.
