2024-08-21 VSE Workshop

I think Blender should also add sound mixing, to attract producers and let them contribute to dev fund. The ground has already been laid with that GSoC project.

Text editor can be turned into a screenwriting tool also, to benefit from that WGA union money, and speaking of VSE it should be able to assemble presentations, there is a strong large community of students out there.

But most of all its a shame that the Spreadsheet editor is that weak and abandoned. Hell, can you imagine the money flow in dev fund if middle managers start contributing?

1 Like

Not going to lie, I wouldn’t mind if the Text editor supported Markdown rendering (even as an addon), lol.

Slightly back on topic, I think it’d be great if every place where there is a Text-type object (text object in 3d view, String Node in GN, Text strip in VSE) it would support bringing in a text datablock from the text editor. It makes a lot of sense to keep blocks of reusable text around in a project.

4 Likes

A concern I have with moving VSE data out of scenes is how well coordinated the effort would be with other efforts to set up some kind of higher-level, project-scope, settings and tools. It’s a solid idea, I would absolutely love to see more decoupling on this front, but it’s also something large enough to merit its own set of workshops. I’d hate to see a situation where a stopgap solution turns into an obstacle for later iterations.

2 Likes
  1. EXR images dwaa with high compression could help to reduce size alot

  2. +1 for otio for I/O

On OTIO, the python option is not very stable, whereas the option in C seems much more solid and it has paved the way for implementations in ex. DaVinci Resolve and Premiere, and might be a relatively simple thing to implement in Blender.

Thanks everyone for the comments! Good suggestions. But just to keep expectations in check: while it would be nice to do everything that is needed, realistically right now amount of people working on VSE sums up to… about one (i.e. pretty much two people, both part time).

  • VSE vs compositor, nodes, etc. – yes, as mentioned, that is generally wanted. No concrete design in how exactly that would look just yet.
  • VSE data outside of the scene, or some other ways to edit sequence timeline while being able to tweak some other scene at the same time – yes, that is part of the whole “larger picture” section.
  • Hardware acceleration (NVENC etc) – yup, under GPU section.
  • VSE “primary goal” – sure, while primary goal is “tools for making stories”, there is nothing that prevents you from just stiching several movie clips in VSE. No one wants to take that away or make it complicated.
30 Likes

This is so great! Awesome big scope breakdown and insights! :fire: :purple_heart:
I particularly love the removing of caches and complexity which is of current dubious usefulness.

Congratulations on the merge of ConnectedStrips :tada:

As a geeky note, it’s very exciting to see a good grip on the pixel processing pipeline too.

1 Like

It’s been over 20 years since post-production software started treating 3D scenes as just another node, with the possibility of mixing them and controlling render parameters from another node.

I think VSE should be the root, the Compositor a modified Strip, and 3D scenes objects with their own timeline.

This image shows how rendering could be controlled from the Compositor.

As nickberckley says, the audio section also needs more care.

Thanks aras_p for all the improvements added.

3 Likes

Feedback:

  • I am glad storytelling UX is been thought about with the power of the sequencer!
  • Glad to hear ideas of integrating the compositor toggling or loop-back to the sequencer is also been thought of.

About the “pinned” sequencer timeline, I have been using code from the Spas Studios branch adapted to 4.3, using the hack with another custom fork of Blender.

The “3D Sequencer” when using the full feature set and “switching” scenes from a single timeline is… a godsend and I can’t live without it. A more performant or elegant solution I would be totally happy to work in! Working with the adapted 3D Sequencer from Spa Studio has saved me so much creative time and upped my game hands down.

4 Likes

Well, that’s disappointing, and quite concerning as well. It seems like BF is not aware of how their userbase actually use their software. Very interesting indeed.

If the sequencer content edit was moved out of the scene data structure becoming a data-block, and the individual strips also could become data-blocks, maybe the current node systems only would need sequencer-data-block input and output nodes (single strips as data-blocks could almost work out of the box)?

Imo, it’s better to integrate the VSE in the existing editors, instead of making new editors (ex. a node editor) in a VSE friendly version. And paradoxically, this is properly best done, by moving the sequencer out of the scene data structure. Ex. if the sequencer edits and individual strips were data-blocks, they could be listed, organized and imported into the Asset Browser, and trimmed here before 3-point-edit adding them to the sequence. And the same way could a strip, as data-block become a texture in the 3d space etc. Or a text/image-data block from the text/image editor could become a text/image strip.

So, another way of thinking about the VSE, is as an editor(hub/assembler) of data-blocks(potentially created or edited in all the other editors of Blender), and the edit of the sequencer itself should be a data-block, independent of the scene, so that could be used anywhere in Blender which takes a data-block as input. That would make the VSE an extremely versatile tool…

and basically become…

wait for it…

The blender of Blender!
:joy:

5 Likes

That sounds like a beautiful dream if it becomes true.

Well, the Asset browser handling image, video and sound data-blocks including thumbnails and drag and drop into the VSE was working, until it was removed, that is.

1 Like

It wasn’t removed. When the Asset Browser was first made all datablocks were allowed, but due to time constraints only a few were implemented “properly” over time (Material, Collection, Geometry Nodes, World, Pose).

You can allow all data-blocks to be assets by enabling “Extended Asset Browser” under Experimental features:

all_asset_types

11 Likes

That’s fantastic to know! :open_mouth: Thank you! I hope they can be properly implemented in the not-so-distant future :grin:

1 Like

So, it is still somehow floating around on the river Styx. Any plans on what coast it is going to end up at? For using the Asset Browser to organize VSE footage, afair only thumbnails for movie strips, were missing (and of course an option to either import A/V directly into the Asset Browser, or mark the VSE imported footage as data-blocks(which I did an add-on for)). I actually did a patch for movie strips, but it was undecided whether to commit it or not(so that is long dead and gone):
MC_asset_preview

D’n d of movies worked too:
ID_dnd

But it still serves as a good example of something as simple as having VSE strip content marked as data-block will make the VSE much more integrated in Blender.

7 Likes

i just want vse to use the gpu. glad that’s finally getting talked about more.
give us gpu support and we’re good. I could honestly care less about all the other features.
i just want this thing to run above 5fps. It’s not usable rn.
Im hoping you guys have the funding required to make these changes… hopefully soon.

I want Blender VSE to FULLY replace Davinci Resolve for me. For 1 reason.
Interface Customization. That and it’s just way easier to edit videos in Blender, aaaand ive been using blender for years. All we need now is GPU support for VSE to actually be usable. In my case, I’m getting 5fps sometimes 0fps in the sequencer with my core i9 CPU and 96 Gigabytes of RAM. Can’t wait for blender to update VSE with GPU support so we can actually use VSE for real. It’s a brick right now.

I think you want performance, not necessarily “gpu” (why? because there are other factors that make VSE slower than it could be, not just “lack of gpu whatever”).

It would really help have test cases where you get 5 or 1 FPS playback, just to see which parts are slow! Right now I am testing on VSE cases as used/needed by Blender Studio, but none of them are as slow as 5 fps. So yours presumably uses some different features or different settings, or something else from VSE that make it way slower. It would be good to know what is slow, i.e. have some blend+media files that roughly show your setup and are reproducibly slow.

Today VSE is roughly two people part time, so about “one person” working on it. Just to set expectations.

8 Likes

Ah… wasn’t aware there’s just 2 people working on this… damn. I might as well move back to davinci and just try to deal with the forced layout. or just wait a while for vse update.

I believe im getting 5fps due to my color data. It’s HDR video at 3440x1440 60fps footage. But VSE can’t seem to run it reliably.

If the vse used the gpu we’d be set rn. No more performance issues I’d imagine. As far as I know, current vse just uses a single core on my cpu… So yeah, I just want better performance to be honest, VSE is good as is rn. All it needs is decent performance.