Suggestion: Make the Sequencer contents into data-block, which can override the switched to Scene-Sequencer contents

[tintwotin]

It is quite a limitation that you can’t switch scene without switching the contents of the Sequencer too. If this was possible you could use the sequencer to switch cameras and scenes in the 3d view, which would be very useful for ex. grease pencil or animatic storytelling.

As I understand it, is one of the reasons for the Sequencer being inside scenes is that the render settings also are nested inside the scene data structure, but instead of moving the entire Sequencer data outside of scenes, maybe just the content of the sequencer could be converted into data-blocks, like the texts in the Text Editor, the images in the Image Editor, or the videos in the Movie Clip editor. If this step is too complicated maybe the content of the Sequencer could be set up as a override function, where a dropdown could be used to determine the current sequence and not change, when changing scene.

Codewise I guess the VSE copy/paste code could more or less be mirrored, so users could load/save the sequencer contents(all strips “copied” and then saved), or “paste” (all edited strips data loaded from disk and “pasted” into the sequence).

The dropdown(like the data-block selector in the Text Editor header), should contain a Current Scene setting, so everything will work as it is now, but if a new sequence is added or loaded, the Current Scene edited sequence data would be stored in ram, so the selected edited strip data can replace it. The same thing would happen when changing scenes, if Current Scene isn’t selected in the dropdown, then selected edited strip data will override it, while the current Scene contents is saved in ram.

I don’t know if this will solve the recursive problems of rendering 3d view scene strips of the current scene, but basically Blender could work around this when rendering, by always copying the current vse scene to a new scene only used for rendering and delete it afterwards.

[JeroenBakker]

Personally I think using an id datablock for the sequence editor is something we should do. In short term there is one feedback loop that we should be limiting (scene strip to self). In the long run this is just a step towards a more flexible pipeline.

Next to the render setting also check color management setting and if there is any breakage in this workflow. Not sure how to handle caches….

Note that this change isn’t as straight forward as it seems. It impacts scene data, animation system, depsgraph, python api, blend file breakage, Id management. I would advice to look at these areas as well as some effects artists and addon devs as well.

[tintwotin]

@JeroenBakker Thank you for sharing your thoughts of this topic.

There are some great doors opening if the feedback loop could be worked around ex. switching cameras in the 3d view from the Sequencer’s scene strips(2.79):

Previz Camera Tools [Add On] for Blender

Or having a sequencer channel input node in the compositor(The compositor-Up-fork):

Using Compositor-Up's Sequencer node and the Video Sequence Editor Transform Tools add-on - Blender

Or roundtrip strips in the compositor, the tracker, 2d editor(masks), 3d editor etc.

Or having sequences and strips show up in the Asset Browser and Outliner for organization, render-versions or as drag and drop elements. Or as swapping data-blocks between ex. storyboard, animatic, various versions of renders.

[ok-what]

I really like the idea.
I have some questions about it:

From what I understood from this post and the one you wrote on right click select, the sequencer would be completely separate from the scene. I’m guessing it would also have its own render settings, such as the frame rate and the resolution. Where would these be shown? In the properties panel the same way other data blocks such as the world are shown?

What would the render image and render animation buttons do? Would they render the sequence or the scene, and how would you choose which one to render and what sequence to render?

Would you be able to select a specific sequencer data block to be used for rendering a scene (eg. from the post processing section)? One that would be used when rendering a scene. This could make it nicer for camera switching, but I’m not sure if it’s the best way to go about it.

How would backwards compatibility work? Would each scene with a non-empty sequence be converted to data blocks?

Edit: added a question

[tintwotin]

Well, those questions should be up for discussion. So, you might as well try to answer those yourself, however, the devs are properly the only ones who can answer them in detail.

As for render settings, IMO, they could also be moved out of the scenes, be exposed in the sidebar of the file browser pop up, have a selector to determine what should be rendered ex. 3d view, compositor, or sequencer. The render should also be possible to call without the pop-up, so it goes straight to rendering, so the functionality would be the same, as it is now.

I’ve tried out having the a/v media in the VSE converted to individual data blocks, IMO, this is quite useful, since they would be useable in the Asset Browser and outliner. If you want to try it out, you can grab a copy of Blender 3.1, enable the experimental setting for the Asset Browser, select media strips in the VSE and run this snipped: https://gist.github.com/tin2tin/0da859896432245fd15cfe960fdd9ee9 in the Text Editor, then the clips should show up in the Outliner(broken heart icon) and in the Asset Browser. From both places, the A/V data blocks can be dragged and dropped into the VSE. You’ll also find the material as a selectable option in ex. the Movie node. Drag and drop of ex. movie clip, sound, or image into the node editor, could also auto-add an input node.

Drag and drop of media data blocks into the VSE:

[ok-what]

My thoughts on this as a reply to @Bachadam 's post here: Proposal: Using compositor nodes on VSE strips - #28 by Bachadam Read it first for context. I replied here since this is a more relevant thread.

The “principal VSE” as I understood is similar to @tintwotin’s idea. Independent of the scene, with the ability to apply node based effects etc. on the strips. I like that idea.
I don’t understand why the old “Scene VSE” would be left in though, if it’s just a more limited version of the “principal VSE”

If it’s there to have a link between the Scene and a specific sequence, I propose this:
The checkboxes in the “Post Processing” section are replaced with a dropdown:

The option in the dropdown tells what to render when Render → Render Image/Animation is pressed, just like now.

• When “Scene” is selected, sequencer and compositing is disabled.
• When “Compositor” is selected, it gives you the option to choose which node group to use, like in the previous proposal:
  
• When “Sequencer” is selected, it gives you an option to choose which sequence to use.
  

For “Compositor” and “Sequencer” whether or not a 3D scene is rendered depends on whether the sequence includes a 3D scene. This works the same way as it does now, but the difference is that you can now choose which Sequence and node tree to use.
A sequence that has Scene strip “Scene.001” with the input as “Sequencer” can’t be used as the sequence for the same scene (“Scene.001”). That would be the only limitation. The same scene can be used if it uses the camera as the input, as that won’t lead to recursion.

Individual compositing trees and sequences (regardless of Scene) can be rendered with the play button (like in @tintwotin’s proposal):

How the resolution and frame rate would be defined for separate sequences is something I’m not quite sure about yet.

[tintwotin]

@ok-what
IMO, users need a much simpler solution, which I think should be possible if the original suggestion here was considered.

As I understand if a scene can only have one output at a time, and since the 3d view, the sequencer, and the node editor all are embedded in the same scene, recursive problems will happen when the output of one of them is used as input in the other which is why getting the output of ex. the 3d view or the compositor of the same scene into the sequencer is disabled.

The suggestion here is that the Sequencer data should be moved out of the scene data, so you can switch scenes without switching the content of the sequencer and the problems of recursion will not happen. Like you can switch Movie Clip, Text, or Sound data-blocks without switching the scene. This will mean that you can work in the Compositor on a specific strip in the Sequencer master edit, having both open in the same workspace. Or you can edit durations of scene strips pointing to cameras in the 3d view from the full sequencer edit and still in the same workspace edit the 3d content(without having to switch scenes) - this is how camera switching is done in ex. Unreal, Unity, and everywhere else.

There will be problems with the render settings embedded within the scenes, for simplicity in use, these settings should also be considered moved out of the scenes.

[ok-what]

@tintwotin

The suggestion here is that the Sequencer data should be moved out of the scene data, so you can switch scenes without switching the content of the sequencer

Maybe I misunderstood or didn’t explain it well. My suggestion above assumes that the compositor and sequencer data is separate from the Scene like you proposed. It only has to do with what gets rendered and when.

What do you mean by original proposal?

[tintwotin]

As a workaround, @Shubhampatilart found out that making a “linked to” copy of the current scene, and assigning that scene to a scene strip in the current scene, will allow for edits of the objects in 3d view, to also update in the vse preview(if the cache is switched off and vse is constantly updating by ex. playing).

This version of scene strip tools will automatically assign the linked scene copies: https://github.com/tin2tin/scene_strip_tools/tree/linked_copy

And it will allow camera switching in the 3d view from the sequencer:

This method of a “virtual scene” only work for the 3d view, since the objects of ex. the vse or the compositor are not IDs or Data Blocks.

[filedescriptor]

I started to work on this about a week and a half ago. Just uploaded a work in progress patch here: ⚙ D16276 [WIP] Video Sequence Editor: Remove dependency on scene. There is not much to be tested for now as it barely compiles, but I will keep updating this the next days.