I’ve been jotting down some thoughts and proposal this past week on the Long Term Support (LTS) plan as set out by @Ton.
I’m formulating most of these on my LTS process wiki page, but I’m writing here to ensure we’re discussing this in the open, and to have feedback while we improve our understanding of the implications of LTS on Blender development.
Please read the wiki page to get all the text, but I’m condensing the information here in a few points as a TL;DR;
A new LTS release each May, good for 2 years
In August 2021 we’ll be running no less than 3 concurrent releases (2.83 LTS, 2.93 LTS, 3.0 non-LTS)
API needs to stay unbroken for 2 years in an LTS release
Triaging and bugfixing will need to be done against all current releases (in 2021 that’d be 2.83 LTS, 2.93 LTS, 3.0 non-LTS)
Documentation needs to be properly branched, managed and maintained with the three concurrent releases in mind
Find good release cadence for patch releases (maybe once a month to keep big pile-up from happening).
Anyway, let me know your thoughts!
/Nathan
p.s. I’m going to cross-post on bf-committers as well, but hope that we can centralize the discussion here on devtalk.
I’d like to see better defined what and what will not make it into an LTS update, run of the mill bug fixes are obvious, but lets say nvidia comes out with a VTX 4000 (or whatever the new shiny toy of that year will be called) will we add render kernels? or will that be master only? Will the LTS branches receive library updates? if no, what if a library update is required to resolve a bug? those sort of things
Well, since it is LTS I’d be leaning towards adding support for it in a patch release. I wouldn’t make it a rush job though, especially if we can agree on doing regular, “light” releases.
Perhaps we can involve such players in the market in helping realize support on the code side for that.
Since our libraries tend to be living in our own garden I think it would be useful to include bug fixes that require updates to our dependencies. Again, since I’m hoping for a frequent but “light” approach for patch releases it could be handled over the course of 2, perhaps even 3 cycles before it gets out to the public.
Naturally this all requires we are dilligent with our fixes and implement good testing practice so that we don’t release regressions instead of patches
Maintaining APIs should be carefully planned for no less than three concurrent releases.
Well, this is really a matter of not committing bugfixes that break the API, and if the bugfix is critical finding another way to fix it. There’s not a lot of planning involved in this as far as I know, it’s just a rule to follow.
A bug should be fixed first in the oldest release branch, then propagated through all necessary branches. In 2020 it will be in two branches, but in May 2021 it will be three branches (2.83 LTS, 2.93 LTS, master - work for 3.0 non-LTS).
When this was discussed here internally with Ton, Dalai and Sergey, I believe the agreement (and condition for this to be approved) was that LTS would not be a significant burden for the regular development team.
Someone would be given the role or be hired to be in charge of the LTS, and would then be responsible for backporting important bugfixes, creating tests where needed and managing the release process. Sometimes that might require assistance from other developers, but it would mostly be something they don’t think about day to day.
I have created a proposal diff D7319 that introduces a new part in the versioning scheme. In BKE_blender_version.h there will be a BLENDER_VERSION_SUPPORT_TYPE , which is either empty or set to lts . When set to lts on the splash screen LTS will be suffixed after the release version, separated by a space. The LTS string will be used also in the System Information output. Finally it can be part of the release package file name.
Putting LTS in the version is fine with me. But it’s still undefined what the LTS version numbers will look like as a whole. Will there be an “2.83a LTS”, “2.83b LTS”, and so on? Or will the number be different?
Will the LTS get daily builds on the buildbot, and if so what does the full version number for that look like?
Will the 2.83 release immediately have LTS in the version number, or will that wait a bit until we have e.g. 2.83a and we are a bit more confident about stability?
I had not given it any thought but once i saw the 2.83q in the screenshot, it does strike me as worrisome, the wiki proposes a patch every month, with 2 years worth of updates that gets us scary close to the end of the alphabet for a/b/c numbering, I mean throw in a few out of band emergency patches and we’re in trouble.
I would add this as a reason to notremove the subversion number from the footer. In an ideal world, we could rely on users to accurately state the version that they are using, and to accurately label their screenshots when uploading them in reports. In this ideal world I would never have to merge duplicate bug reports, or tell users that they have to upgrade their drivers. As we don’t live in an ideal world, knowing what version of Blender is being used at a glance is helpful.
People sometimes start using Greek letters when this happens. For example, we get a lot of hurricanes where I live. Once a there’s a named storm for every letter in the alphabet, they start using Greek letters to name them instead.
So the numbering goes from x.0 to x.7 for each series & there won’t be x.8 & x.9! wouldn’t that cause a confusion at least from the users pov ,when we see 3.7 then the next one 4.0?
Not day-to-day thinking, but when a bugfix is being considered I’d hope devs also ensure they keep an LTS in mind.
Completely forgetting about a release is not feasible at all, but having someone keeping tabs on non-LTS work and see how it fits in LTS, then work together with the relevant devs - that I think can be usable.
This approach that was apparently internally discussed brings another issue to light: what to consider for LTS? Personally I’d say that this would include also work that improves on existing features. Which I think is something the users could expect from an LTS release.
Doing that IMO doesn’t constitute a burden, but is rather a logical and natural result of maintaining and supporting a codebase. I hope we can instill in our devs more and more an attitude of creating code with a certain longevity - not just code and forget…
To not confuse matters I’d stick with the same version number for at least major and minor. What we use after that is really open. I’d rather have a micro number of sorts, say 2.83.0, 2.83.1, etc. We already have a subversion number, so maybe we could use that instead as patch release denominator?
I would assume so, part of my patch (but completely untested) is to add lts into the package name. We could re-introduce the rcN modifiers in both version and package name to communicate this information.
I don’t see why we would wait with LTS in the name until a later patch release. Either we support the actual release and do patch releases for 2 years, or we don’t.
I would argue that after a year an LTS becomes pretty stable (wishful thinking? Healthy optimism?) and we might see time between patch releases space out. With a monthly cycle there isn’t much need for out-of-band emergency patches. Maybe when code was introduce that fries computers after a second, but hopefully our patch review process picks those out already before we are in an LTS… But who knows.
Aye, the system-info.txt at least carries subversion, and the system-info.txt is probably something we should make ‘mandatory’ for bug reports. Having the complete version number in the splash screen would help IMO as well.
I’m not too worried about that. With a two year cycle I think users quickly catch on when a major new release is around the corner. Am I being too confident in our users?
I don’t know what this means exactly, but this sounds like a burden.
The wording from the blog post is “important bug fixes and updates for new hardware”. No other improvements, no unimportant bugfixes. Basically the same rules as we use now for making a or b releases.
If the scope can be expanded to include a few more things I don’t necessarily mind, but it really should not be a burden on developers.
If we can’t refactor some code or have to make a suboptimal design decision because it makes porting back changes to the LTS harder, I think that would be unacceptable.
We don’t want to encourage developers to write code that needs to be rewritten again soon. But regardless we have code that is in dire need of rewrites and we plan to keep making big design changes.
I don’t think that would work well since it increases rather arbitrarily and will be at an arbitrary value at the time of release. We could make the current subversion number strictly internal and have another one that is increased in a more predictable way.
Really the choice is between using letters or numbers, either can work but there needs to be a decision.
In the past we’ve used “rc” to mean something like “beta”, where we know the build is not actually a candidate to be released and will get more changes. And we intentionally stopped doing that since the name was not really correct.
Imagine the names for the releases and daily builds:
2.83g LTS and 2.83h rc2 LTS
2.83.7 LTS and 2.83.8 rc2 LTS
2.83g LTS and 2.83h beta LTS
2.83.7 LTS and 2.83.8 beta LTS
I think the last one is most clear? Or did you have something else in mind?
It’s still a mouthful though, we could consider not making “LTS” part of the version name. How to display it depends on the context, but for example on the splash screen you could for example have “long term support” under the version, maybe with a little less constrating text color.
Right, then we need clearer guidelines on deciding what is important and what is not important. I still find it hard to determine where the distinction is drawn. On the one hand I can see that it could mean crashes only, but then again users may disagree here.
Right. I meant this more the other way around, pretty much like:
IOW not so much looking backwards but rather forwards.
Naturally, that is only to be expected. But once a certain part of code diverges greatly from a current LTS it still is likely that a developer needs to look at the LTS, probably even more so than when code and design haven’t changed much.
If this were a vote I’d vote for numbers (and not based on subversion of fileformat).
I agree. the last one is the clearest and I was hoping someone would suggest it.
Sure, but this is not a problem for screenshots and system-info.txt.
I have no strong opinion on how it is shown, as long as it is clear from an install what version is being used, preferably with as few clicks and hoops as possible.
I strongly agree with @brecht here, things as they are worded in the proposal right now do not feel acceptable nor in line with what was suggested in early discussions. IMHO, after a few months, both branches (master and the LTS) will have diverged quiet significantly, and bug should then be fixed primarily in master.
It is then the responsibility of the LTS maintainer(s) to check whether that fix can be applied as-is on the LTS, if the general idea behind the fix is applicable even though adapting the patch would require some work, or if it is simply not doable in the LTS.
‘master’ developers should be of course available for advice and help, but the idea that in two years, they should first fix a bug in two years old code and then port it twice across hugely changed code bases is a huge red flag to me.
In fact, I would rather treat LTS as a separate code base altogether, with its own workforce and own bug tracking etc. Then if some issues are close enough in both codes, all the better. But I would not expect that to happen really often. Just have a look at the differences between 2.80 and current master…
Bug triaging is also a concern actually. We already barely manage it in current situation, adding more very different releases to the equation is not going to help either.
Regarding versioning, I don’t have a very strong opinion but think I like the 2.83g LTS schema the most. I would stick to letters, this is much more in line with what we have been doing already since ages for corrective releases, and it will avoid the confusion with internal subversion numbers.
I’m afraid i cannot find a good reason to do this, for the minority of users who would like this.
LTS is a request in disproportinal user base demand.
If we think about keeping futures, it be far easier to maintain versions optionally in new branches.
So you wont triple the bug support demand (2.83. 2.93. 3.0).
But then what does it mean to maintain futures in new branches ?.
Blender settings should allow to keep using specific versions, (ea various dll version).
Think modular, have options to use a specific version of something.
The old cloth sim, the old particle system, the old cycles noise texture or the old cycles engine etc.
Code wise Blender could modulary contain those versions in its install base, and still progress without LTS versioning.
As a bonus people will be able to use old ways of doing certain things, for example keep the old particles, but still bennefit from a faster render engine.
LTS is typical for people who think in server software release cycles.
Blender is not a server application, it can be solved by modular design.
Even for render farms this would be easier i believe to be able to do certain things at a certain level of blender but be able to ask your end user to use the latest (fastest) render engine, or possibly later override render engine with a network enabled version of cycles.
I’m using blender for quite a long time now, i cannot remember drawings couldnt be openend / converted. OK we cannot do Blender internal render and Blender game engine, but if work depended on that i think time stood still.
LTS is something requested by studios, that together contribute a significant amount of funding. So it’s not disproportional in that sense.
I estimate that the maintenance cost of this would be an order of magnitude larger than LTS releases, especially long term. I don’t think it’s practical.
A big reason we removed Blender Internal and Game Engine is that they were constraining what we could improve in other areas, and making it harder to implement certain improvements. For example the material system of a renderer has implications for viewport previews, texture painting, and file import/export.
Modular design is good, but it’s no magic bullet. Everything still has to work together and the glue that binds together those modules needs to keep evolving as well.
The first goal should be to get to an LTS of a year, not two, so you don’t have overlapping LTS. In fact, the initial plan should just be an attempt to obtain this once. So a one-year LTS is released with 2.83, followed by about 10 months of struggle, then a couple of months to decide if you want to do it again with another LTS released in May 2021. You could decide to not do it again if the experience was horrible, try another 1-year, or extend it to two if it was all a breeze.
Emphasis mine. If we really can have an LTS maintainer then this sounds valid to me. I don’t mind we approach the LTS from this perspective for now, but it does mean we will need someone (or two) who do this. I’d still expect some communication with other devs, as you say as well, especially when codebases have diverged significantly.
Indeed not, but we still have to take this into account. If and when we do have an LTS maintainer this could probably be coordinated through this position with the rest of triaging team.
Probably yes. I am not married to letters or numbers, but I do think that having numbers is nicer from semantic versioning point of view. Again, I am open for either approach.
This may be an important point to consider for all of us. I do like the idea of doing this step-wise.
I also wonder if the addition of LTS should only be done in concert with changes to releases in general.
We could perhaps have “Experimental”, “Stable”, and “LTS” branches. The current nightly builds of both Experimental and Stable being always available for download from BO, without any traditional version releases ever. Only LTS gets periodic releases like we do now, but on a long schedule like once a year.
A patch submitted gets reviewed and approved for Experimental, but is not closed. After a specified amount of time the module owner could decide that this patch did not cause problems in Experimental so it is approved to move to Stable.
The amount of time that a fix/feature/change must exist in Experimental before moving to Stable would change depending on the release schedule of LTS (and the nature of the patch). So if just after an LTS release a bug fix could go from Experiment to Stable very quickly while a feature/change would take a couple of weeks. But as we approach the next LTS release the times are lengthened. A minor bug might remain in Experimental for a couple weeks while a feature/change has to stay there a couple of months. Then this more-stable Stable becomes the next LTS release candidate.
With a “Release Ahoy” only happening once a year with each new LTS, I’d pick a date that gives us maximum exposure so perhaps timed for SIGGRAPH in mid-July. So that LTS would include no features/changes added since May and not even any hastily-added bug fixes. What is then added to the LTS after release? No new features or changes at all. Only selected bug fixes that have been tested in Stable for a period of time.
I’m with Harleya on this one. I do see a point in maintaining an LTS version of Blender, but to overlap two LTS releases while working on the latest Blender patch at the same time seems a bit extreme when it is a brand new idea for Blender that has never been done before. LTS releases should really not overlap unless you know for sure that you have the resources and manpower to do it.
Also, wouldn’t it make more sense to do a 1 year trial of an LTS release where you are committed to making it work and then rethink the plans of what to do next when you have figured out what works and what doesn’t? Saying that the Foundation is going to commit to 2 years of fixes to an older Blender version before you’ve figured out the logistics and rules and put them to practice may complicate things further.
