Layers maniphest

I think the part of what he is talking about is how to address the collection from pipeline scripts, right now collection cannot be reliabley addressed from a script because the do not have a predictable addressing schema . There adrees change through their lifecycle based on their position in the higherarchy .

So basically you can create a script that will send your assets to the same layer, that will survive the layer being renamed or moved in the outliner I think?.

a static predictable internal address for collection would make this system more easy to manage in a pipeline.

Perhaps a scheme similar to what is used in networking could help by using three disgnation for collection

  • it’s name for the user it could be named anything
  • It’s dynamic address aka IP which can change with reordering in the outliner, can be used for manually managing the collection from the UI by the User
  • a static internal address aka Mac address that doesn’t change at all, with maybe a certain range reserved to be assigned by pipeline scripts for special purpose layers and the rest of the range is just incremented for when it doesn’t matter

By doing it that way you can have scripts that will implement cqd with any hacks

The “types of contrast” is interesting because I’m teaching someone Blender and one question I got is “why are there two render menus”.

What she was referring to was the Render menu and the Rendering workspace, which looked the same to her.

A few hours in, she was still confused which was which, and still had trouble differentiating between what is a menu and what is a workspace, because looking at the top of the screen, the only difference is an ever so slight value difference on their backgrounds.

1 Like

I’m not sure that this can be determined as a major problem - Collections system was designed to bring maximum flexibility, and name-related addressing is quite enough to work with any kind of static contextual separation. Mac-addressing for scripts can be solved as collection name prefix, to avoid manual name changes, but such scripting is pretty much rare case anyway.

However, in business, content management systems assessment concerns not only flexibility but also management effectiveness. The dependence is almost linear - the simpler the system, the simpler the management it requires, the more flexible the system, the more complex management mechanism is required for effective system management.

The problem is that this was completely ignored when developing the collection management system.

Management system is all about daily usability of a system. The management task to be solved is a business problem, and sound like that -
make urgent corrections to the scene containing 500 collections of the 6th level of nesting, in which all possible types of visibility and several view layers are used, made by several artists who are inaccessible now.
That’s just common situation in business.

Currently we’ve got a system with 3 types of visibility control for every collection (EC, VV, DV):
RTOs

Two of them are structurally-dependent (VV, DV), two of them are interconnected (EC, VV), that brings insane amount of flexibility to the system, but it’s not even possible in Blender to confidently and non-destructively (temporarily) isolate a Collection in order to immediately verify its contents without loosing current collections setup, to properly configure or check up what will be rendered, and so on.

Actually, almost any action in outliner currently leads to loosing your current setup, that makes managing complex muly-user projects incredibly tough.

So it’s all about making Blender’s vertical (hierarchical) navigation usable for complex projects management, immune to large business stress tests cases and suitable for teamwork.

3 Likes

I did a poor job of explaining things — I was talking about a tag-based filtering system, not UUID’s. UUID’s are more of an asset engine thing, although I can see how they’d play a role in collection instancing.

Tags are a form of descriptive metadata that can be used to identify the objects they’re attached to. If you’ve rated a song in your favorite music player or attached a color label to an image you’ve used tags. If you’ve used Instagram, Twitter, or Steam, you’ve used tags. If you’ve used Windows Explorer you’ve (hopefully) used tags. They’re pretty common, and extremely useful.

Tags aren’t limited to ratings and keywords; you can use just about any data type you can think of with the right UI. For bit array tags the ‘right UI’ would probably look like a block of toggles, similar to Blender’s old layer system, paired with a switch to turn the tag filter on and off.

I assumed 1D was talking about a bit array tag system because, frankly, it’s the only way I can see to efficiently implement what he’s suggested. Tag based filtering can work both across and within hierarchical structures, and it can provide a way to set and restore viewport visibility without touching the rest of the outliner. Bit array tags themselves are a byte weird, but the general framework could be incredibly useful.

yeah this is also confusing to me

A bit about types of systems and their purposes.

  • Collections system, as a Common layer system from 3dsmax or Maya, is intended for handling static context - it is needed for complex scene setup.
  • Layers/QCD (Quick content display system) - for dynamic one - it is needed for quick access in complex constructive multirefrenced modeling.
    So, they cannot be compared.

If you played modern games, like Fallout 4, Far Cry or Dying Light, the inventory slots there works like a Collection system, and the Weapon selector slots - like Layers/QCD.
They work together, and Weapon selector is just a dedicated (accurately numerated) part of inventory.


It is impossible to say that weapon selector is better than inventory, or inventory better than weapon selector - they are different content management systems that behaves differently, and are needed for different purposes.

This way, comparing Collections and QCD systems is like comparing how much animation in Dope Sheet is better than in NLA editor.
So, we can perform complex constructive multirefrenced modeling in 2.79 with Layers/QCD, “or” make complex scene setup in 2.8x with Collections, instead of making both in 2.8x, what caused the community split between hardcore modellers and artists.

Layers/QCD system’s goal is speed of combinational access.
It has got one of the best realization of dynamic context management system in modern CG, even modern games takes that concept, it has got the best realization in comparison with other software.
The problem of 2.79 is not that there are Layers, but that there are only Layers.

Collections system’s goal is capacity.
In terms of production processes any content management system is characterized at least by two parameters - flexibility of possibilities and efficiency of management of these possibilities.
Collections system have to be checked by the laws of static context management system, and it have a lot of issues to fix and improve to became a good static content management system, as it have low efficiency of management of its possibilities, even in comparison with similar systems in other software.

3 Likes

In fact, the QCD system is the answer to the question “how complex model can you build in your software without getting ripped”.
We are currently trying to reconstruct the QCD system to make Blender compatible with the multirefrence modeling workflow.

2 Likes

Thinking of making proper video about Content Management systems.
It will take time, but hope to come out with nice explanations.


may 31

Finally, the ability to clone a view layer, including while copying a scene, has been added.
https://developer.blender.org/D6862

It was completely unexpected that it is possible to create a scene setup system without being able to clone the actual scene setup. To satisfy production needs we were forced to create a phantom mode that works like temporal view layer, to explore scene setup.

3 Likes

Well, after 2 months of production, video is finally ready.

6 Likes

@1D_Inc Having you “fight” for layers and snapping tools is a guarantee that one day we will have something really powerful in this blender area :fist:

A lot of the problems mainly come from two decisions that make things very complex to manage. Much more so then in other 3D applications.

  1. The ability to have the same object in multiple Collections. Not to mention hierarchy related issues with this.
  2. Merging rendering & scene management in one setup, aka the View Layer.

I do like the way Collections makes a ‘standardized’ way of Collecting objects inside a scene into one convenient ‘package’, especially for linking or appending, but for a lot of users it became the preferred way of organizing the scene. Which is wrong imho, and I think the ‘Scene’ should be default in the Outliner on startup. Not the view layer.
CAD based models with tons of hierarchies is a good example why the Scene should be default.
Even for a experienced 3d user it makes a confusing combo of scene management & Collections for grouping (and unfortunately… rendering)

For splitting & rendering out your scene, most applications split out the scene (organisation) and render setups. Yes, there is overlap, but they are mostly separate identities.
It makes for a much cleaner organisation, especially in larger groups.

I applaud your work and video, but it only shows me the flaws in the current system, and another addon should not be the way to go here.

The Collections concept should be revised to make it less confusing and more manageable.
For now, it’s all over the place, and a bit of a mess.

Hi.

This was called as Cumulativity issue (at 2:48).

This issue consists of several issues, like RTOs that are not stored in a View Layer, which is also part of RTOs relationships complexity issue, and so on.
Basically, current Collections realization is a system that consists of issues and problems that are in some kind of weird balance.

Yes, it was discussed there as well, in Grouping tread.

Thank you. I made this video to have the ability to show and explain that flaws to developers. Also,
Such basic systems as hierarchical/management systems should never be solved as an addons. We are making addon as a prototype because of despair.

Undoubtedly.
But direct conversations with devs didn’t brought the result - hierarchical issues are always hard to explain. Much harder than “cloth sculpting brush” issues.
Therefore, we are trying to make proper explanations of the disadvantages of the current system and to design possible solutions.
As an addon, we are very limited, but we are doing our best.

3 Likes

Hi,

I agree on all parts, even the part that is is - really, really - hard to address these things with the devs.

It’s also my opinion that things like the outliner should be a task for the in-house developers, not just one guy, who is doing this as a GSOC project.
Don’t get me wrong, I salute Nate for all his hard work, - and - willingness to listen to the users.
Otherwise we would still be stuck in 2.80 Outliner country… :wink:

1 Like

Hi, Collection Manager dev here. :slight_smile:

Yes, this adds greater flexibility, and so a greater chance of misuse, but I think this isn’t necessarily a problem and in some situations it might be useful.

I actually think the collection/view layer paradigm works pretty well. Yes, there are some problems with the implementation, but the overall idea, I think, is a good one. If you think about it, it makes sense to allow stuff to operate on a whole collection, whether for scene setup or rendering, and it’s also nice to have the ability to have multiple configurations for scene setup (and this is essential for rendering). In the end, you want an encompassing “layer” for both scene management and rendering, and since the toggles can live happily side by side I don’t see a problem combining the two. You could separate them out into Scene Layers and Render Layers, but I think this adds needless complexity. I will say that the Restriction Toggles are not presented in the clearest and most efficient manner and could do with some improvement, and their tooltips could certainly use some work (as indicated by two separate patches currently sitting in the tracker), but overall I think they do an okay job.

I will say that grouping should not be handled by collections, but by actual groups, and this whole issue could definitely use work.

I’ll agree that for some things it would be much easier/better to have them as default in blender, but for a lot of things the Collection Manager does very well. And, of course, a default implementation could have better visual indicators of states and stuff like that (however, it is possible to draw everything yourself in OpenGL from an addon, so technically adding better visual indicators is not impossible. It would be a lot of work and quite possibly not a good idea, but it’s not impossible). Even if everything were added to the Outliner I would still want a collection managing popup in the 3D View. And for things like QCD, I think it wouldn’t be done much differently for default blender (they might even do it as an addon).

I will agree, though, that in principal, the basics should not have to be solved by addons from outside the main development team.

Well, I’m not despairing yet! :slight_smile:
And an addon gives us a wonderful playground for testing out and designing advanced management systems which can then, potentially, be added to blender (at the very least there will be a battle tested reference of what we want, that we can point to).

Well, devs have a lot on their plate, and while that doesn’t mean they’re perfect, or can’t be annoying at times :wink:, we should remember that and try to cut them a little bit of slack.

From reading through this thread, and others like it, there seems to have been lots of discussion on why improvements are needed or that things are “a bit of a mess”, but less concrete discussion of exactly what’s needed and how it should behave. As a developer, I may not always understand at first why something is needed, but if I’m told clearly how something should behave, it helps.

Of course sometimes the devs just don’t get it/agree with us, and there’s not much we can do about that.

The QCD system actually came about because I was reading this thread and I knew that the devs weren’t understanding what @1D_Inc was talking about, but I did, and I knew that it could be solved as an addon and would incorporate well with the Collection Manager, so I made a prototype and we developed it into something wonderful.

I wouldn’t say that we are very limited in what we can do. There are limitations, but I think we have done very well so far. I have seen some truly amazing things done by addons, so I would say that the skill level of the developer is more important than whether it’s an addon.

2 Likes

Nice example here can be Group Pro addon, that adds such desired common groups, but makes files unusable without that addon. That’s why common groups realization is important - it spreads as a virus because of popularity, forcing you to buy this add-on when you meet the scene where it was used.

And that is nice! But I was talking about our company.
For example, as a project manager I working with lots of talent artists, which can make beautiful visualizations, but make a mess even with a limit of 20 layers. The problem is that now they have an infinite ability to spoil the scene. That’s why between mess and limitations business always chooses a limitations.

Collection Manager addon is a miracle.
But the problem is that building a business taking into account the likelihood of miracles is quite difficult)
We’ve spent too much time and efforts trying to explain issues of a system,
and faced with complete despair at the end, before you started this addon.
I would never have thought that what it was doing would be possible, it is truly amazing.

Thank you so much for this!

3 Likes

That is the main feature of this system

True, cumulativity is one of the features of the collection system.
As well as there were no presented proper tools for handling it.

Cumulativity has come from 2.7 layers and groups (2.7 grouping system is actually a tagging system, not a grouping system), so presenting it to a static context systems is quite experimental.

So there are no examples of tools that can properly handle it for static context separation across the industry. This is a gap in management tools concepts, it is not known whether such tools can even exist.

It is not a problem to handle cumulativity in 20 layers, or incorporeal (with no RTOs) tagging system, but when cumulative objects are scattered across 200 collections, this begins to be a serious problem.

3 Likes

Interesting. Yes, groups should definitely have a proper default implementation.

Ah ok. I wasn’t sure who you were talking about so I decided I’d just add my view to the conversation.

Yep, everything’s perfect until you add people into the mix. :wink:

Interesting.

Of course. You can’t depend on miracles. Nor should you have to.

I just try to tell it like it is and make things that are flexible and work well. And I’m not just fishing for more compliments, although they are, and always have been, appreciated. :stuck_out_tongue_winking_eye:

1 Like

Speaking about limitations in business - it depends on business scale.
For example, AAA gamedev have very cruel naming conditions, textures type and models quality restrictions, otherwise it will be impossible to control all that content, that generates this industry properly.
The entire BIM in architectural industry is build around limitations and restrictions, so you can’t create whatever you want anymore, like in regular modeling - everything is structurized there by default, to avoid structural mess.
And so on.

2 Likes

Current common groups realization discussions I found:

  1. Blenderartists
  2. Devtalk
  3. Devorg