Enums are used in many places in Blender and ideally we would use them in even more places (replacing integers).
Unfortunately, enums have a problem that is not very obvious at first: Different compilers are allowed to use different underlying integer types for the same enum.
For more details, read this post by @LazyDodo: https://developer.blender.org/D9736#242235.
This has two main consequences:
- One cannot safely serialize and deserialize structs that contain enums (with simple memcpy), because the struct layout might not the same. Therefore, we cannot use enums in dna.
- A function with an enum in its signature that is defined in C, but called from C++ (and vice versa) might fail at run-time. This happens when the C and C++ compiler use different underlying integer types for the same enum.
Given that more C code is converted to C++ code in Blender, the second problem could mean that we can’t use enums in a significant number of function signatures.
That is because we have a lot of functions that are defined in C and called in C++.
The goal of this thread is to decide whether we can safely ignore that second problem in the context of Blender and to document our decision.
Here are the two options we should pick one of:
- Work under the assumption that all pairs of compilers might use different integer types for enums (as is allowed by the C and C++ standards). This means that we can rarely use enums at all in interfaces that are used by C and C++ code.
- Assume that for the C and C++ compiler combinations that are actually used for Blender, the selected integer types for an enum are always the same (so e.g. gcc and g++ are compatible, but not necessarily gcc and msvc). This allows us to use enums in all pretty much all function signatures in Blender.
In either case, enums must not be used in dna structs. Allowing that would force us to make greater assumptions about compatibility between compilers.
My opinion is that we can choose the second option, given that we haven’t run into related problems in practice yet (afaik).
I’d still like to hear the opinion of a couple more core developers, just to make this decision a bit more official.