Blender LTS

Hi everyone,

first of all congratulation on the first LTS release. I have been asked to maintain Blender LTS during the upcoming period. I have written https://wiki.blender.org/wiki/User:Jbakker/projects/BlenderLTS/ProcessDescription on the wiki explaining a bit how it will be organized.

The process is written so it won’t effect regular development. During the upcoming period the process will be tested and stabilized. There is also a task added in phabricator to execute and monitor the process (https://developer.blender.org/T77348).

Any input is welcome!

16 Likes

Congratulations! :tada: This is huge

Hey Jeroen,

Thanks for writing up the proposal. Here a couple of things:

“08:00 to 12:00 AMS time”. Please change to an actual timezone, maybe add several different ones with their respective times.

This proposal together with the one posted on code.blender.org don’t give a clear picture as to what goes in and what not. In particular I feel that the proposals lack how triaging is handled in whole.

I believe that the current proposals combined fail to address regressions. A regression is something that also should be included next to critical issues. It would be good if there was a clear documentation on what actually is a critical issue.

The passage “LTS reports are treated the same as for previous releases – if already fixed in master, the report is to be closed.” is troublesome. This should be handled differently: if a report for an LTS release is valid, even already fixed in master, it should still be tagged for the LTS release in the case of regressions and critical issues.

The triaging and testing process should be adapted to include LTS as well. The following flow is what at minimum should be done: 1) test against latest LTS - if valid regression or critical accept for LTS, 2) test against upcoming blender-vx.xx-release - if valid issues, accept for blender-vx.xx-release, 3) and finally test against master.

The phabricator task section is insufficient for triaging purposes.

In your bf-committers message, and with online discussion it is stated that “The process is written so it won’t effect regular development.”. I have said it before, and I will say it again: work on LTS __is__regular development. I can’t shake the feeling that the current proposals and approaches, along with existing mindset, is insufficient to do proper LTS.

I’d like us to reiterate the current proposals, and make sure the entire process becomes an intrinsic part of how we develop Blender - not a tacked on afterthought.

edit: some text I forgot to add

To help with the tagging of tasks before they end up on the parent task I would suggest adding a project per LTS release. That way we aren’t restricted to having only one milestone tag per issue (2.90 vs 2.83 for instance)

I think having a parent task alone is insufficient for triaging and tracking issues, especially when tasks are still being considered for LTS before being added to the parent task.

/Nathan

1 Like

Hi Nathan,

Thanks for your input. I fully agree that the definition of critical fix must be defined better. It is also something I am struggling with. During the pilot program we follow similar rules to how corrective releases were handled. The goal of the pilot program is to see how we could organize this in the future without influencing regular development.

In my opinion stability and quality are the core principles of LTS. Development done specifically for LTS isn’t what we should be aiming at especially during the Pilot as that introduces new bugs and would lead to double the effort for bug fixing. Something that is currently already taking 2+ days a week of all core developers. Needless to say that it doesn’t fit in the budget of a pilot program.

Adding a Phabricator project would help in this case. I will check the details for that. Point that I get is that the Pilot program needs better communication and clearer explanation.

I think this is the incorrect mindset for development. LTS is regular development as any other development. Already approaching LTS as if it were the ugly baby no-one should have to touch with a pole of any length, except for the one lucky/unlucky (choose depending on view) dev, is not going to yield good results for the pilot.

Instead we should look for ways to ensure LTS is an intrinsic part of development attitude for all developers involved. Note that this doesn’t mean all developers would be working on LTS. But it should mean that LTS is a fully accepted product we actually want to support throughout its lifetime.

Development on Blender regular and LTS should in the first place be about the user. Although making lives easier for devs is naturally useful to achieve satisfaction for the user in every aspect of Blender.

LTS being an intrinsic part of regular development or not to me is rather vague. It has to be part of regular development to some extent, but I think it can be limited in practice. I don’t have a good overview of how other software projects handle it, but I think for example Linux LTSI is handled by an independent development team.

There is already a big cognitive load for developers, in terms the number of tasks to do, and processes they have to understand and follow. We can’t keep adding more and more rules that every individual developer needs be aware of. In my opinion we are already at a point where if you add some complexity you should really simplify something else at the same time to keep things manageable.

For example we now have more complicated branching with the new release schedule and bug triaging got more complicated. On the other hand we also did things like add make update for easy branch switching, automated release builds, hired more bug triagers, and hired Aaron to write the user manual so developers don’t have to. But I think on balance we still need to try to simplify and automate the common tasks more, not complicate them.

2 Likes

When it comes to the LTS specifically I think the LTS maintainer tracking the master branch without involvement from other developers or bug triagers handles most of the work. That master branch commit log is much more reliable than getting dozens of developers and volunteers to follow more complicated rules for bug triaging, or for the LTS maintainer to keep track of what is happening in the bug tracker.

The large majority of fixes for LTS will automatically end up in the commit log. I wouldn’t even try to track bugs for the LTS until a fix is committed. If something is a serious bug or regression, it should be marked as high priority and tackled soon regardless if it will end up in the LTS.

The one exception is critical bugs that exist in the LTS but not in master. This might be because the bugfix was not backported, in which case the LTS maintainer can be notified or a developer can backport it themselves. Or because the bug exists in code that was removed or changed in master, in which case the bug can be marked as high priority and handled by the module owner as part of regular development. This I expect to be relatively rare.

I don’t have a particularly strong opinion if a task or tagging or some other mechanism should be used by the LTS maintainer. But I don’t think that triaging or fixing for every bug should have the overhead of always testing with LTS. Only when it turns out that a critical bug exists in LTS but not master.

4 Likes

Hi all, congrats on the first LTS release!
I have a couple thoughts and questions about the LTS process. I hope this is okay.

What is the relationship of addons with the LTS release? Will fixes for them be allowed in? Because addons are less likely to break blender will lower priority fixes be allowed in, say medium to high priority fixes?

According to https://wiki.blender.org/wiki/Process/LTS only severity 1 issues would be allowed in as opposed to the severity 1 & 2 issues that would be fixed in a corrective release for a normal release, and the corrective releases would potentially come much later for an LTS release than for a normal release. This suggests to me that the LTS release would end up with a greater number of bugs for a longer time period than if it were a normal release. I don’t know what companies expect from an LTS release, but I would treat it as a regular release for maybe, the first 3 months and then move to a schedule of regularly spaced bug fix releases, and include normal severity and up issues in the scheduled update. When bugs are reported from the “Report a Bug” option in blender the blender version is automatically included and should make triaging fairly easy (it’s too late for this release, but if LTS could be added to the version string it would help to further differentiate and make triaging even easier)? Since the LTS release just continues on with the branch created for bug fixing in bcon3 I think the process for bcon6 could be similar to bcon3 and then at the appointed date it could be tagged and released.

This all sounds fairly straightforward and easy to me, but I could be totally wrong, so take it with a grain (or mountain) of salt.
Just my two cents.

Would using BF Blender (2.83)'s Workboard help?

(fig.1 a mock example)

Since there are developers tagging bugs with 2.83, I think it would be interesting to have this as the back-end of lts management. It gives a clear visual indication of what exactly is going on, plus can be easily sorted and managed.

I’d prefer something like this too over the current maintenance task. But for LTS’s it would be better to use a new project because otherwise we can’t tag for different milestones (2.83LTS and 2.90).

1 Like

Damn missed that, could do it with sub-projects but new tag means most flexibility.

So with a new project we can have the commits tagged and show up in queries for their version (eg in the left menu) and this would allow for different major releases of lts to have the same corrective fixes tagged and organised in their respective workboards.

The workflow:

  • Developers tag tasks with lts(s) that they deem appropriate
  • Maintainer can accept or reject
  • Once fixed in master, fix for lts is committed, tagged and shown in corresponding version fix

Are there any glaring problems with this approach?