Some ideas for GSoC VSE projects - we should aim for small well defined ideas - so which idea would be the best suited?
First I want share, what I see as the top 3 of challenges for the VSE, basically I think the VSE should become the hub for realtime story development and virtual storytelling inside Blender:
The biggest all time challenge is INTEGRATION, simply because without it, the VSE go no raison d’etre. Or in other words if the VSE is directly integrated in the 3D View(In and Out), meaning you could use the 3D View for previewing the video VSE contents and control the 3D contents from the VSE(ex. camera and scene switching etc.) with the lightning preview speeds of eevee, no one would argue that during the storytelling process, you should render everything and edit it in an external NLE, at least not until you would have to assemble high quality renders, which could be argued, could be better done in ex. Davinci Resolve. With better VSE integration, comes also round-tripping of strips in the Compositor, but basically make sequences and strips available on a frame change basis from/to the other Blender Editors. In order to improve the integration a necessary step is to move sequences out of scenes, so changing the scene doesn’t change the contents of the sequencer.
The second biggest challenge is PERFORMANCE which concerns speed of playback(in 3D View, preview of video, scene strips(playbasts), multithreaded, cache optimized, GPU usage, fast image scaling, fast and invisible proxy handling/cache to disk) and quality(unbroken color-manegement, 10+ bit color, high quality image scaling etc.). This being second, since the final export quality could be handled externally(but would need a proper export option like OTIO), but it is very close to the number one, simply because the slow playback of ex. video on a first run will scare off many first time users.
The third biggest challenge is the TOOL SET & THE UI that comes with it. With the toolbar finally integrated it is now possible for proper icon based tools, but it needs serious planning to make it well designed, user friendly, UI integrated(on strip drawing/interaction) and (splitscreen)preview of ex. trim and slip/slide operations etc.
The seems to be a wide support for a better integration of the VSE into the other editors:
- A better integration of the VSE with the Compositor, like roundtripping strips in the Node Editor, is one of the most requested topics in the Sequencer tagged posts on Right Click Select.
- A better integration of the 3D View with the Sequencer for ex. a storyboarding workflow, has been discussed much. There are multiple ways to address this ex. the playback performance of Scene Strips is very slow(a eevee rendered scene running at 30 fps in 3d view runs at 0.3 fps as eevee rendered Scene Strip) and/or live camera and scene switching in the 3d view from the sequencer(3d View Strips?), which would need a separation of sequences from scenes.
- Allow VSE to control frame handling of media assets elsewhere in Blender, like Movie Clip or Image Sequence inputs of texture or compositor. This allows you to time clip start/stop and offset in context with other animation.
- Develop for a seamless from storyboard/previz(or even screenplay: https://github.com/tin2tin/Blender_Screenwriter ) to final film workflow, without rendering to file until everything is finished.
- There is a general need to improve the playback performance of the VSE, but can it be boiled down to a GSoC suggestion?
The VSE UI/toolset:
- On strip drawn operations like fade, volume rubberband, crossfade, thumbnails, and modernize the strip and handle drawing?
- Channel headers with options to mute/hide/lock(like NLA).
- Move Strip Effects into Modifiers Tab.
- Populate the newly implemented Toolbar with essential tools.
- Integrate basic image asset manipulation like scale, translate, rotate and boundary clip/mask.
- Industry standard 3 point editing with source, rec, in, out, overwrite, insert, lift, remove and channel headers to select/deselect what channels to operate in(would also need a separation of sequences from scenes(for ex. more playhead positions and sequences in the same window)).
- Auto preview of trim operations(including dual screen), strip parenting and other strip editing related operations(populate the toolbar), overhaul of the VSE API.
- A bin system based on the File Browser as Assert Manager(or is better to wait for a general Assert Manager solution to be implemented?).
- Integration of the OTIO library which will allow the Sequencer to import and export in industry standart formats like aaf.
Proxy and ffmpeg:
- An overhaul of export, proxy and ffmpeg operations(including exposing ffmpeg to the API). Currently, export and proxies are generated single threaded(very slow) and the UI is not user friendly.
- Implement support of VST, LV2 and Open FX plugins?
- Color handling overhaul. Currently the color handling seems broken and slow.
- 10, 12 etc. bit video?
General VSE ideas: devtalk.blender.org/t/video-sequence-editor-a-list-of-ideas/10343
VSE Quick Tasks: devtalk.blender.org/t/quick-hacks-for-the-vse-a-list-of-suggestions/10341