2024-03-14 Triaging module meeting

[lichtwerk]

Triaging Module Meeting

Biweekly Triaging module meeting for planning and coordination.
Folks on the payroll should attend if possible, others are invited as well.
In these meetings, we can keep us up-to-date on everything triaging, discuss recent important issues and see if we can bring the module forward.
I believe it is also good to see human faces once in a while :), so there is always room for some personal stuff if appropriate.

The meeting will be on 2024-03-14T12:00:00Z. It is open for everybody interested to join the video call (link below).

Links

• Video call google meet for now, we can try jitsi as well if people prefer
• #module-triaging chat channel
• live meeting notes

Attendees

• TBA

Topics

• followups from last meeting

  • number of untriaged reports is still a bit high, goal is to half the number the coming weeks
  • improve upon issues with unavailable hardware
    • Philipp: MacBook arrived
    • Hardware List has been ported to Hardware List - Blender Developer Documentation
  • feedback regarding our triaging infrastructure
    • gitea issues
      • have these been reported to Issues - blender-projects-platform - Blender Projects ?
    • bug reporting wizard
      • initial doc gather thoughts/ideas
  • labels / issue titles / commit messages
    • Germano put up suggestion for fine-grained labels
      • feedback from Nodes & Physics was… sceptical
  • automated crash reporting
    • some feedback (in chat), discuss further
  • triaging metrics
    • Thomas updated script (split up by module), Grafana intergration TBD
  • first round PR review
    • news?

• recent important issues that are somehow hard to triage

  • if time permits, go over specific issues

Meeting Notes

• Follow up from last meeting
  • Number of untriaged reports is still high, continue to work on reducing this number.
  • Improve upon issues with unavailable hardware
    • Philipp’s M2 based MacBook arrived. This helps with testing Mac specific issues.
    • We may be missing hardware for specific bugs. We’ll evaluate this as they appear.
  • Feedback regarding our triaging infrastructure
    • Gitea issues
      • These have not been reported yet? (Double check with Ilya)
    • Bug reporting wizard
      • Current notes on design document seem fine.
      • Germano created a prototype Blender addon for this.
        • It creates extra work having to maintain two bug reporting interfaces, and dealing with how this addon will integrate with Gitea, and specifically Gitea/BlenderID authetication.
        • This system has too many downsides at the moment. We are unlikely to go with this approach, but will keep it as a possibility.
      • Germano explored Gitea issue report formatting. (Changing the bug report form currently in use)
        • Report formatting has some limitations (E.G. Input fields can’t be dynamically shown/hidden). We should make a feature request to Gitea developers upstream to see if they would like to implement this feature, and if so, what the timeline for this is.
        • Even with the current limitations we can roll out some useful changes in the near future.
        • Modifying the Gitea issue report formatting has many benefits (Easy to setup, avaliable to all users making bug reports, etc).
        • This is the prefered choice for the bug reporting wizard.
      • A report will be created on Gitea to track this further.
      • A pull request is progress for an intial prototype.
  • Discussion of general triaging practices.
    • Should unconfirmed but important bug reports be marked as high priority?
      • Ideally no. The triaging team should try and reproduce and confirm the issue first.
  • labels / issue titles / commit messages
    • Labels
      • Germano created a list of possible changes and additions to labels.
        • Feedback from the Nodes & Physics module was that there was too much overlap and confusion with the proposed changes.
      • We should reach out to more modules and developers to get feedback on how they use labels, what changes they would like, and if they want changes at all.
      • Ideally we should have more fine grained labels (specifically Interest labels).
      • There should be some clarification on which areas each module manages. Module owners should update their module pages on this. This will help triagers figure out which module labels to give bug reports.
        • We will contact modules to encourage them to update their pages with the relevant information.
      • Some module labels are unintuitive or hard to figure out for some reports (E.G. There’s a VDB rendering issue, without looking at the code we don’t know if it’s a issue with the render engine or VDB file loading). However these types of reports are uncommon.
        • Adding more labels (E.G. A volume label) may help here. But we’ll need to discuss this with module owners.
    • Triaging metrics
      • A script for extracting triaging metrics has been written and the output of it regularly gets put in the blender-coders chat.
      • This script will be commited to a Blender repository in the near future.
      • Grafana integration will be worked on later due to work around the Blender 4.1 release.
    • Automated crash reporting
      • With this system there is the danger of loosing proper bug reports from users as they submit a semi-automated crash report and believe they’re done.
      • At the moment there isn’t much useful information in Blender crash logs that can help track down the steps to reproduce the issue from crash logs alone.
        • Extracting more information from the user/Blender and putting it in the crash logs would be useful here.
      • Arnd wants to comment on the idea. They may have useful information/ideas/or insights.
      • We’ll need to disucss this further before taking any action.
  • Extensions platform
    • The triaging team will triage these bug reports.
    • We still need to figure out which parts of the extension platform we’re handling (E.G. We won’t handle bug reports about extension, but we will handle bug reports about the platform). This will be discussed in the future.

The meeting was cut short (90min) due to Philipp having other arrangements. The planned topics that weren’t discussed because of this are:

• First round PR review
• Recent important issues that are hard to triage

Following meeting

The following meeting will be on 2024-04-04T11:00:00Z. Again it will be open for everybody who’s interested.

[Alaska]

@lichtwerk

Bug reporting wizard

@modmoderVAAAA has created a task to track this system here: #78 - Triaging Wizard - blender-projects-platform - Blender Projects

---

Brecht has currently expressed interest in avoiding integrating/packaging the triaging wizard with Blender, which I believe we agreed upon in the meeting. Brecht has also expressed the interest in integrating the wizard into Gitea as one of the better solutions. This comes in the form of modifying the Gitea issue template, or modifying the Blender foundation version of Gitea to provide a better bug reporting interface (E.G. Add Javascript support to the Gitea bug reporting page, or creating our own).

---

Due to this interest, and the work and resources provided by Germano over the last week, I decided to look into modifying the Gitea issue template to find out how useful it is, and what limitations it has that gets in the way of improving the bug reporting interface. My testing can be found here: Alaska/bug-report-testing - bug-report-testing - Blender Projects

1. Descriptions are easy to ignore.

When adding a section to the Gitea issue template, you can add a description to explain what kind of input you’d expect. The font size and font colour of the description make it easy to ignore. This can likely be fixed by changing the Gitea theme.

Notice how the description is dark and small on a dark background. This makes it easy to ignore.

2. Pre-filled text boxes count as filled out

When you add a area where a user can fill in information, you can mark it as “Required”. Meaning the user HAS to fill it with information before they can finish the bug report. This feature is great and would help out with bug reports where people don’t fill out any useful information.

The issue is that if you pre-fill out a text box with some information for the user (like the image below), and the user doesn’t modify the information, then the text box is considered “Filled out” and the user can make the bug report without filling the text box with useful information. We, or preferably the developers of Gitea, could modify Gitea such that if the input text information is the same as the default, then the report won’t be filed.

The other option is use place holders (grey text that shows up to give an example of what you should write, see the image below). But this place holder text disappears as soon as the user starts writing something. Meaning users may not use the formatting requested (E.G. A user may fill out the Blender version section with five versions, but not mark which ones are working or broken because they ignored the place holder and we didn’t fill it out with text for them to use as reference.)

These issue with the place holder text is made worse by the fact the descriptions are easy to ignore (see point 1). (We could include examples of what we want in the description, but if it’s easy to ignore, then it’s not useful)

3. No dynamic content hiding/showing

This has already been discussed. Check boxes, input fields, and descriptions, can’t be shown/hidden dynamically. What you add to the issue template will be visible 100% of the time.

4. All check boxes show up on the report

With the issue template there is the option to add check boxes. One of the things I wanted to use check boxes for was making sure that the user knows that whatever they share is public, and they need permission to share it. For example:

For this case, I want a check box that the user has to read and agree too before they can file a bug report. But I don’t want the output of the check box to be visible on the report once it’s been made. But there’s no option to control that. The check box will be visible on the bug report no matter what.

Maybe the Gitea developers could improve this aspect? It’s useful for situations like this, and situations where people are agreeing to terms and conditions.

5. It's hard to do some forms of formatting

I would like to add some checkboxes to the issue template to help guide/encourage the user to do some testing. And by ticking or not ticking the checkbox, we as triagers already know what the user has done.

The issue is that some people may not know the steps required to do some of these things (E.G. A user may not know how to reset to Blender’s factory settings). So I would like to include extra information on these check boxes to help them:

However these guides and extra notes will show up on the final bug report, which isn’t useful.
So I have to use other formatting techniques to convey this information, and it can get messy.

6. Extra useful tools are missing

Controls for minimums and maximums on the length of a text input don’t appear to be accessible (We don’t want users pasting there entire crash log into a text input, or just deleting everything and replacing it with a single letter).

Text hints don’t appear to be available (You know how some websites will have text that’s underlined, and when you hover over it a text box will appear explaining something? I call those text hints). This could be useful for work around the issue I brought up in point 5.

Each input that accepts multiple lines of text can have files attached to it. But these files all end up at the bottom of the bug report when it is made. Controls over which areas can accept file inputs, what file formats they accept, or even if file previews are automatically added to the text field would be useful.

7. The Gitea issue template is too limited for more complex ideas

One of the proposed ideas for a triaging wizard was to add a tool to help guide users through bisecting a issue. The Gitea issue template is not designed for these types of tasks and so integrated a bisecting tool straight into the issue template is not easy or is not pretty.

There is the option to add a link that takes the user to another site that helps them with bisecting.

Disclaimer: These are just my thoughts. What I express here does not represent the views of the Blender foundation, or the triaging team. And some of the features I desire may not align with what others want. Along with that the triaging wizard is still in the ideas/prototyping phase. So anything that gets mentioned here will likely change.

---

With the Gitea issues template in it’s current form, we can make some small adjustments that could help users. For example: Break the system information, Blender versions, short description of issue, and steps to reproduce section up into seperate text fields and provide clear descriptions.
But some other ideas that we may want to explore are hard to implement with the limitations this system has. We can either try working within these limitation, work on fixing them in Blender foundations Gitea, or see if the Gitea developers would fix them upstream. Or transition to something more custom.

---

Anyway, these are just my thoughts and observations and it would be great if others could share their thoughts and ideas. Whether that’s here in text, or in the next module meeting.

[Alaska]

During discussion around the triaging wizard (I believe this just occurred on the design document), there was talk about integrating the triaging wizard into Blender and including tools in it that could help the user identify the potential cause for issues in more complex files.

With the preference for the triaging wizard to NOT be integrated into Blender, it basically eliminates that idea. But maybe it becomes it’s own project? So we have the triaging wizard which is web based and brings various benefits. But there’s a second project, “triaging tools” built into Blender (either as a bundled addon or built into Blender) that could be used by users. But this will obviously need to be discussed in some form or another before taking any action.

I also feel like we should get the triaging wizard under the way before we start work on these “triaging tools” unless there are enough free resources among team members to do both at the same time. But that’s just personal opinion.