How to report complex production bugs?

I often encounter more complex scenes in commercial production scenarios that are full of interdependent issues. In current case, the scene has about 6 objects with complex modifier stacks which take several dozens of seconds to calculate. This results in Blender freezing for about half a minute.

Refreshes of these modifier stacks are caused by seemingly random things, for example turning off a collection that has absolutely nothing to do with those objects, or adding a new material onto an object completely unrelated to those objects. Or duplicating it. There are numerous different action that seem to trigger update of modifier stack on completely unrelated objects. At some point, the .blend file becomes more or less unusable because doing pretty much anything freezes Blender for over half a minute.

The issue is these scenes are large in size, and also under an NDA, so they can not be shared freely. They also do not manifest a single issue, but multiple interdependent issues. Blender’s system of reporting bugs seems to be completely incompatible with this class of issues:

  1. It is required to always report only one thing per report, so all the different things that can incorrectly trigger the modifier stack refresh would have to be reported as separate issues.
  2. There is no mechanism for sharing files that are under NDA
  3. There is no mechanism sharing large files
  4. There is no mechanism for interaction with the bug triage staff in real time, since issues of this complexity can rarely be showcased by just a linear list of steps.

This is not the kind of bug where you can upload a simple blend file with a default cube and provide repro steps #1-5. This is the kind of issue where you’d ideally want to share your entire complex production file privately with some developer, who would dedicate bigger chunk of time into investigating and fixing the whole group of these issues rather than just patching a single occurrence. I suspect these freezes are caused by numerous unintended dependencies in separate areas of Blender rather than by just one core issue.

This is the class of real world issues which prevent Blender from handling complex production scenarios which most people simply do not bother to even report because the process of reporting them is too cumbersome and unclear. This unfortunately means this class of bugs will probably be present in Blender forever until something changes about the bug reporting process.

8 Likes

In my opinion handling complex reports like this requires as significant effort by both the triaging team and the reporter. As someone who has worked on the triaging team before, I’ll try address your points as good as I can.

I often encounter more complex scenes in commercial production scenarios that are full of interdependent issues. In current case, the scene has about 6 objects with complex modifier stacks which take several dozens of seconds to calculate. This results in Blender freezing for about half a minute.

This sounds like the issue is easily reproducible from a triaging perspective, but there needs to be a good reason why the modifier stack evaluation that causes freezes in combination with heavy use of modifiers is a bug. Performance issues are generally not triaged as bugs.

Refreshes of these modifier stacks are caused by seemingly random things, for example turning off a collection that has absolutely nothing to do with those objects, or adding a new material onto an object completely unrelated to those objects. Or duplicating it. There are numerous different action that seem to trigger update of modifier stack on completely unrelated objects. At some point, the .blend file becomes more or less unusable because doing pretty much anything freezes Blender for over half a minute.

This may be a consequence of Blender’s design that you are not aware of and doesn’t necessarily have to be a bug. Performance issues can have a variety of reasons and addressing them can be quite complex. Based on your description I cannot say whether or not it actually is a bug, but you could ask developers of the relevant modules if the behavior you’re seeing is expected. If it isn’t, then we will try to find a way to address this.

(Note: In my original answer I misread this paragraph and therefore assumed a different cause)

It is required to always report only one thing per report, so all the different things that can incorrectly trigger the modifier stack refresh would have to be reported as separate issues.

There is no need to file different reports if the underlying issue is obviously the same. The important part is that there needs to be at least one way to reliably reproduce the bug.

There is no mechanism for sharing files that are under NDA

That is correct. Depending on the issue, you may be able to replace the meshes with different ones that still allow to reproduce the bug which would avoid the NDA problem in this instance. If that doesn’t work, you should try to take a look at the original mesh and check in what way it is different to the replacement (non-manifold, degenerated faces, topology, …). This may already provide a starting point for further investigation.

Since the file should be limited to parts absolutely necessary to reproduce the problem, it is usually possible to remove or replace content that is covered by the NDA. I do understand that this does require a significant amount of work from the person preparing the report though.

There is no mechanism sharing large files

On the bug tracker itself this is intentional for two reasons:

  1. Most bugs really don’t need large files and the reporter should try to narrow the cause down as much as possible instead of handing the triaging team a file that contains hundreds of things completely unrelated to the issue. This makes triaging unnecessarily difficult and time consuming. It is in the reporters own interest to provide a simple file that allows to reproduce the issue to keep the triaging process efficient. There may be cases where it’s really not possible to remove or simplify anything, but those are quite rare. In those exceptional circumstance it’s a good idea to have a chat with the triaging team to coordinate. If the issue is only reproducible in that very specific project, but it allows to reproduce the issue reliably, then it might still get accepted.
  2. Disk space is limited and the infrastructure couldn’t handle it if everybody starts to upload files that are multiple gigabyte in size.

If it is absolutely necessary, then there are ways outside of bug tracker for hosting such files. However, if the triaging team accepts those reports is decided on a case by case basis.

There is no mechanism for interaction with the bug triage staff in real time, since issues of this complexity can rarely be showcased by just a linear list of steps.

That’s not entirely true. While we don’t provide 24/7h phone or chat support, but we can all be contacted on https://blender.chat/ and we have a dedicated channel #blender-triagers. This channel is not meant for user support, it’s for the internal organization of the triaging team. However, if a complex bug situation needs coordination with the team there is the possibility to reach out either to the entire team or specific members.


In general it sounds like you are looking for enterprise customer support, which Blender doesn’t offer. The good news is that Canonical’s support appears to provide pretty much what you’re looking for.

This means you will either have to do your best to provide us with a manageable example file that you can share without breaking the NDA and coordinate with the triaging team or you / your company could sign a contract with Canonical for commercial support.

4 Likes
  1. In my particular instance. I have for example one object with multiple modifiers which take quite a while to evaluate, and this object has Material A assigned on it. In the scene, there is also different object, not in any way linked to the original object (no hierarchy/parenting relation and no linked data), not even sharing same collection. If I assign material A to this different object, it triggers the modifier re-evaluation on the original object. How do I know whether this is a bug or performance limitation.

  2. I did not mean that I request support. I don’t expect this to be fixed anytime soon before my project finishes. As Blender is free, I definitely do not expect any support at all. What I meant is that I would just like to see at least a chance of these kinds of bugs being fixed in Blender, so I would just like to have a way to report them, so someone is aware of them, even if they won’t get fixed.

  3. In most cases, simplifying these scenes or attempting to recreate the issues from scratch is what makes the bug vanish. That is the whole point. There are many advanced uncaught complex issues in Blender, which are a consequence of this “as simple as possible repro scene” approach. When all you have are trivial scenes to test on, you will fail to identify all the issues caused by some of the Blender features breaking under the weight of complexity.
    To give an example, even since 2.8, up until now in 2.93, there is a bug which causes very frequent crashes to desktop when undoing texture paint strokes with Eevee viewport enabled. But despite it being the reason I gave up on texture painting in Blender altogether, I never reported it because I always failed to reproduce it on a simple scene, yet it always eventually started to occur in complex production scenes I was working on. Scenes I had no way of sharing privately because I didn’t know if there’s any mechanism for it.

  4. I don’t understand how Canonical support would ever be helpful. Even if they managed to identify and fix the issue (which I highly doubt, because most competent Blender developers already work for BF), what would guarantee their fix being accepted to Blender main branch? Especially if it would be something deep like significant changes to dependency graph? Only feasible way would be to rely on Canonical making custom builds of Blender with the fix from that point on, every time there’s a new version. That just sounds preposterous :slight_smile:

1 Like

Yeah, I think many of the people who are actively using Blender in production face similar issues. Often times Blender would just crash, leaving you wondering “Alright, why did you die on me this time?”, and with seemingly random way these things occur there isn’t much to report in the first place.

One way to improve that would be to have more complex benchmarking/testing scenes, so that at the very least some of that “instability under load” issues could be found and fixed

Great, this sounds like a simple and verifiable scenario both for discussing the issue with other developers and potentially filing a bug report.

That is precisely what you are doing right now :+1:. For complex issues, where it’s not clear cut what the problem is, it takes time and quite a bit of communication to get a bug identified. The next step would be to get the opinion from a developer familiar with the modifier stack and dependency graph and then file a bug report or the other way around (officially the latter is the recommended approach).

Naturally, the example should still show the problem otherwise it’s not helpful to identify the problem. The “as simple as possible” doesn’t mean that the scene can only be simple, it means that it should only be as complex as necessary to reproduce the bug. If a complex file is necessary because the bug only happens under these circumstances, then that is fine. It’s about the right amount of complexity, not that complexity isn’t permitted at all. This requires that the user tries, to the best of their ability, to remove anything unrelated to the problem. What we are trying to avoid is people uploading the files with content that has nothing to do with the problem they are reporting.

For instance, if I’m understanding your point 1) correctly, there is no need for a complex production file in the bug report. You could create one object with a compute intensive modifier stack, one material and another object. Even if that is oversimplified and the actual file would be more complex, it likely doesn’t need e.g. the original meshes, packed textures, all of the materials, particles systems, rigs, nested collections, … that would be in the production file. I understand that simplifying a scene can be complicated though.

The case with the Eevee crash is something you could’ve reported on the bug tracker with the crash log attached. If the bug isn’t reproducible for us the ticket will eventually be closed, but the triaging team has seen the issue and we are usually able to spot when similar issues start to appear and we let the other developers know. So if there is a bug like this that likely affects quite some users, developers will take their time to investigate it further down the road even if the initial report wasn’t reproducible.

This is something you should either ask the senior Blender staff or Canonical. I haven’t used their service and my information are also just based on what is publicly available. They certainly wouldn’t try to make design changes to Blender, but from my understanding they would do the bug reporting for you and help your work around existing issues. I don’t know whether they write patches themselves or if they provide custom builds to their customers. Based on the low pricing, I suspect neither.

Commercial support is certainly helpful for companies if they need to be able to get help whenever they need it. The community websites and the bug tracker don’t have any guarantee that your issue will be taken care of within a fixed time frame.

1 Like

2 posts were split to a new topic: Various feedback items

A post was merged into an existing topic: Various feedback items