Guidelines Proposal: Usage of Type Deduction in C++ Code

Context

The proposal below aims at defining what level of type deduction we accept in out code-base. It is intended to be a starting point for discussion (not a finalized, ready to be implemented set of rules).

It tries to balance the pros and cons of type deduction:

Pros:

  • Reduces code verbosity.

Cons:

  • Reduces code clarity and expressiveness (self documentation).
  • Tends to create more verbose and obscure compiler errors.

Proposal

Some General Rules

Do not consider contextual help from IDE as a good reason to remove explicitness from the source code. While they can be very handy, not all IDEs provide the same level of contextual information, and code must remain understandable when such help is not available (e.g. when reviewing PRs online).

auto

In general, auto should not be used unless the type is clearly and unambiguously expressed somewhere else in the same expression, e.g. by using a casting expression.

/* DO:
 * The type of `var` is clear from the casting of the assigned data. */
auto *var = static_cast<blender::Map<std::string, ID *> *>(user_data);
const bool result = my_callback(my_id);
/* DON'T:
 * There is no immediate way to know the type of `result`. */
auto result = my_callback(my_id);

Note that auto should also be used with unnamed types, e.g. to store a lambda (closure type) in a local variable, when there is no other choice to store a local callback (i.e. when a blender::FunctionRef or similar cannot be used).

Another valid usage of auto is with iterators: Usually, the exact type of the iterator does not matter, as long as they are just used through the common iterator patterns.

blender::Array<int, 42> my_array;

/* ... */

/* DO: */
for (auto my_it = my_array.rbegin(); my_it != my_array.rend(); my_it++) {
  int my_val = *my_it;
  /* ... */
}

/* DON'T: */
for (auto my_it = my_array.rbegin(); my_it != my_array.rend(); my_it++) {
  auto my_val = *my_it;
  /* ... */
}

/* DO: */
for (int my_val : my_array) {
  /* ... */
}

/* DON'T: */
for (auto my_val : my_array) {
  /* ... */
}

Lambdas

Lambdas should always explicitly define their return value. This matches expectations with regular functions.

In general, lambdas should not do generic capture of all local variables, but rather explicitly define which variables are captured, and whether it is by reference or copy. Otherwise, one has to read the whole lambda code to find out which variables are used, which ones may be modified, etc.

Non-trivial lambdas should be defined outside of the function call making use of them. A lambda is non-trivial if it does more than a single operation, like wrapping a call to another function.

There are some valid exceptions to these rules though, when the lambda contains most of the logic of the function, which then acts as a mere wrapper around it. E.g. when using certain iteration patterns:

bool AttributeAccessor::contains(const StringRef attribute_id) const
{
  bool found = false;
  this->foreach_attribute([&](const AttributeIter &iter) {
    if (attribute_id == iter.name) {
      found = true;
      iter.stop();
    }
  });
  return found;
}

Templates

Do not rely on type deduction when using templated functions or types, outside of trivial cases. In particular:

  • Templated types (classes etc.) should always have explicit template parameters, unless they have a default value defined.
  • Templated function calls should always have explicit template parameters when they affect their return types. This is consistent with the fact that there is no type inference on returned value in C++.

Example of trivial type deduction:

/* Even though this would seem fairly trivial, the created data type has to be explicitly defined
 * (no type inference on returned value). */
ID *id = MEM_new<ID>();

// ...

/* There is no need to call explicitly `MEM_delete<ID>(id)` here. */
MEM_delete(id);

Example of non-trivial type deduction, threading::parallel_reduce<>:

int val = 0;

// ...

/* DON'T:
 * Nothing in the call below would clearly indicate the type of computed value.
 * Further more, a mistake in one of the parameter type or callbacks definition can lead to many lines
 * of fairly obscure errors. */
// val = threading::parallel_reduce(
//    my_vec,
//    1024,
//    0,
//    [](const IndexRange range, int value) { return value + int(range[0]); },
//    [](const int &a, const int &b) { return a + b; });

/* DO:
 * Here it is obvious what the _intended_ produced value type is.
 * It will also often generate much more concise and readable error messages in case of mistakes. */
val = threading::parallel_reduce<int>(
    my_vec,
    1024,
    0,
    [](const IndexRange range, int value) -> int { return value + int(range[0]); },
    [](const int &a, const int &b) -> int { return a + b; });

What are the guidelines for auto usage in loops? Both the iterator type, e.g. for (auto it = my_thing_array.begin(); it != my_thing_array.end(); it++) { ... } and the range-for type, e.g. for (const auto &thing : my_thing_array) { ... }

1 Like

Good questions!

I would not allow auto for range-for type in general. I think it makes sense to know what type of data you are getting in each iterations since you are likely going to use it directly.

The iterator case is a bit different, in a way the iterator itself is a ‘common pattern’, knowing its actual type is usually not useful as long as it behaves as expected. Would not use auto to store the value referenced/pointed by the iterators’ current step though, so no auto val = *it;.

1 Like

Another case to consider: initializer lists as parameters to function calls. For example:

my_function(5.4, {"hi there", 33, {"bob", "harry"}});

I guess it’s not strictly type deduction per se, but it’s making the same kind of trade offs of succinctness vs explicitness/clarity of types.

In general, auto should not be used unless the type is clearly and unambiguously expressed somewhere else in the same expression, e.g. by using a casting expression.


I would like to propose that auto be permissible for variables that are directly initialized from the unaltered return value of a function:

auto key_frame = get_nth_keyframe(fcurve, i);

The alternative:

bke::animrig::Keyframe key_frame = get_nth_keyframe(fcurve, i);

Doesn’t add any meaningful clarity, IMO, but does add visual noise. If I’m not familiar enough with the function to know what its return type is (at least, well enough to understand the code), then I’m unlikely to be genuinely familiar with the function’s semantics either, and should look up its signature and documentation regardless to actually understand what the caller is doing.


As an aside: coming from Rust, type inference as a concept doesn’t bother me. However, in C++ specifically it makes me uncomfortable because C++ is chalk full of automatic type conversions (a.k.a. C++ is weakly typed in practice). So being explicit about types in C++ is important for verifying/ensuring that code actually does what it looks like it’s doing. But the issue IMO is C++'s automatic type conversions (which we can’t do anything about, unfortunately, and causes other issues all on its own), not type inference.

2 Likes

Putting all the captured variables in lambdas will likely stretch out the first lines of threading::parallel_for, etc., over a number of lines. Why is knowing what is used
inside the body so much different from code that just accesses variables defined
at an outer scope?

2 Likes

Mostly this seems reasonable, I just disagree with some of the rules about lambdas. TBH I don’t think they belong in this post which is mainly about usage of auto.

I generally don’t have the same expectations about lambdas and regular functions; they are different tools for different jobs. Usually with a lambda the context is more visible and the return type is fairly obvious. I don’t think the extra verbosity is worth it in many cases and I would leave this up to a situational decision.

This mostly adds unnecessary boilerplate and words for information that is not important. Lambdas should be easy to create and easy to read, and this doesn’t help that. The default of capturing local variables by reference is fine for lambdas used locally since the compiler is able to optimize and remove unused captures, etc. For lambdas stored for use later as a std::function things are obviously different.

2 Likes

I quite strongly disagree on this point… For me, reading bke::animrig::Keyframe key_frame = get_nth_keyframe(fcurve, i); tells me all I need to know at a general level to understand what this code is supposed to do. Otherwise, I might not even know or remember that there is a bke::animrig::Keyframe type, and actually have to get out of the current code context to go figure out what type of data is returned here.

1 Like

For this specific example the type doesn’t really add too much information. But IMO that’s not generally true. Not all functions are as clearly named and knowing the type IMO makes it easier to read.

edit: to be clear, I prefer the fully typed version for readability.

2 Likes

Actually, this post originated from lambda and template topics, auto was added afterwards, since it has similar implications. :wink:

I do not see how returned value type is more obvious in a lamdba than in a function. Most of the time, e.g. if returning the result of another function call, or the value of a data member, etc., the type of the returned value is not available at all in the lambda code, and one has to go look for the function or data type definition somewhere else in the code to find out.

I’m not sure how to conciliate the ‘explicit capture is too verbose’ stance with the ‘lambdas should be easy to read’ one. If the lambda is simple to read and only contains a few lines of code, then surely it does not need to capture ten parameters, and explicitly defining the few needed ones is not really a big hurdle.

But if you have a reasonably complex function (few tens of lines), using several different lambdas, IMHO having these lambdas explicitly define which data they work on does help global readability of the whole function.

This is not always the case of course, e.g. in original proposal there is an example where the lambda uses generic capture, which I find reasonable: when the function is essentially a wrapper around the lambda, which contains (almost) all the significant logic.

There are likely other valid cases, but I do think that there are also many cases where such generic capture should be avoided, for sake of clarity and readability of the code, without requiring a complete parsing of the lamdba’s code itself.

For this specific example the type doesn’t really add too much information.

It really does though, without further context, you don’t really know what get_nth_keyframe is gonna return, going by just gut feeling, it would not surprise me if this returned an integer representing the actual frame number of the nth keyframe, the fact that is actually returns bke::animrig::Keyframe clarifies things quite a bit.

that being said most ide’s will likely help you and moving my mouse cursor over an auto generally tells me the type.


(my mouse was on the auto keyword)

so i honestly don’t have a strong opinion here, if it makes the code easier to read use auto, if you’re just being lazy and hoping to save some keystrokes i’d prefer if you don’t. ie:

  std::vector<int> mahvec = {1, 2, 16, 18, 22};

  auto vec_iterator_yay = std::find(mahvec.begin(), mahvec.end(), 18);

  std::vector<int>::iterator vec_iterator_nay = std::find(mahvec.begin(), mahvec.end(), 18);

the yay version is much nicer on the eye, and the nay version really doesn’t a whole lot of information.

Kinda feels like this will be one of those “use common sense” issues where we can debate for months what common sense exactly is.

4 Likes

I see explicit lambda capture only as way to manually define capture type or even a move semantic. This was tested that there is no benefits in manual enumerated reference containers like Span as [&, positions, ...] in order to reduce referencing level in a closer so i did accept [&] pattern in general and don’t see any issue with it until you do not do lambda which should live much longer than a function…

Overall lambdas aren’t meant to add another reusable abstraction level, compared to a separate function which is. The set of variables used inside the lambda is not usually more more important than the set of variables used by any other code in a function.

The context available around lambdas is just much higher since they’re part of the function rather than elsewhere.

I don’t believe this is a specific enough way to build a rule about when lamda capture has to be explicit.

1 Like

Totally fair that you disagree, of course! And the more I think about it, the less and less comfortable I’m becoming with it myself in the context of C++. So I’ll defer to whatever the consensus is.

(I still think type inference as a general feature is a good thing, but C++ is… special, and creates an interesting kind of type paranoia that doesn’t exist in most statically typed languages. But that rant doesn’t belong in this discussion.)

1 Like

Same here :slight_smile:

I think it’s a mistake to think of lambdas as just a convenient way to define a new function. Lambdas allow you to encapsulate logic in the context of the function they are in and pass that as callback to some other function, or call it repeatedly, or use the immediately invoked function expressions pattern. This is extremely useful to me in day-to-day programming. I find they make my code better without the overhead of other solutions.

When creating a separate function instead of the lambda, I loose that context. Now I have to jump to some different place in code. And I need to add a name and a function comment to explain the context that I just removed. And all data needs explicit arguments, that I have to manually change when the function needs different data.

I often work in C-style implementation files that deal with quite a few things (deep modules, which is good!), and I constantly get lost in all the functions. This code is a hassle to deal with. Recently I’ve introduced lambdas there, and I find it quite an improvement.

So I think we shouldn’t make this about trivial vs. non-trivial lambdas. Maybe it’s more about context: Code that uses a lot of context from it’s owning scope may use a lambda; otherwise a named function should be used.

3 Likes

Also note that when requiring non-trivial lambdas to be their own function, we cannot make use of captures anymore to inject data into callbacks. Instead, we have to define a callable class/struct like this:

struct MyFunctor {
  MyData &my_data;
  void operator()();
};

And then explicitly initialize the functor and it’s members so it can be passed as callback. This becomes non-trivial quickly.

Or we create a boilerplate wrapper lambda doing the capture, just to call the function. Or, we stuff the data into some “custom data” argument.

None of these solutions are very attractive to me. The wrapper lambda seems okay-ish though if there’s a good reason to use a separate function.

1 Like

I suddenly realize I have a PR that uses a couple "auto"s. Which means that I have to change this one section that uses lower_bound to find an item in a sorted array:

auto weights = std::lower_bound(...

So to this?

std::array<OrderWeights, 731>::const_iterator weights = std::lower_bound(...

This just looks like a mess to me and I’ll have to update that 731 if I add an item to that array.

1 Like

Well, in this case its better to take a look at BLI_binary_search.hh → first_if which is returns an index. In general, i add std::distance(begin(), ...) to avoid such pointers. You also can just derefere it dirrectly if you sure result will be there always. If your input was OrderWeights * then you can also make output ptr of such type (array.data(), array.data() + array.size() instead of array.begin(), array.end() would work but duno if this is better).

For sure I can look at ways of changing that now, and maybe get it closer to the former simplicity that seemed obvious to me. This was a 10-line function where the type was obvious because of the return type and what it was doing. With the “auto” it made sense, but without it is too ugly to live.

In general I just consider less rules to be better and more rules to be worse. When reviewing a PR and a code section is not understandable then ask for changes to make it so. We don’t need a list of rules to justify that.

3 Likes

Regarding iterators, Aras also raised the point in his first comment, I updated the proposal to also allow using auto for them.

I will move forward and write a PR to publish most of this proposal.

The parts that clearly have no consensus currently (the ‘capture all’ behavior of lambdas essentially) will be kept for future iterations.

We can also use std::bind to mimic capture behavior with regular functions? But this is not really the point of this proposal anyway.