Design Proposal: Proxy use in the Video Sequence Editor

Why use proxies?

  • Acquisition and delivery formats is typically far ahead of what common-man’s editing hardware can handle.
  • Playback of complicated and wip 3D scenes, which will not play frame accurate in the VSE, could benefit from rendered proxies.
  • Effect work may need lower resolution proxy files to plat frame accurate.
  • Blender chokes if prefetch cache is slower than the playback rate, with smaller proxies the prefetch can be made faster than the playback.

How do you currently build and preview a proxy for a single strip?

  • Select a strip. (if no active strip is selected the build will fail with no info)

  • Sidebar > Proxy Tab > Check Strip Proxy & Timecode

  • Wonder is this applies to active strip or full selection, now the proxy panel is outside of the Strip tab?

  • Check a resolution(if no resolution is set the build will fail with no info)

  • Check overwrite(if there are existing files the build will fail with no info)

  • Wonder why you would need Timecode?

  • Wonder why you would need JPEG quality, when the proxies are video files?

  • In the panel above Proxy Settings, click Rebuild Proxy.

  • Wonder why you need to Rebuild, while you’re building for the first time.

  • Wonder why you need to use a button in the above Settings panel to build your proxy file.

  • Wait(no multi-threaded processing on the Blender side)

  • Wonder why you don’t see your file in a lower resolution when finished rendering.

  • Open the Preview Sidebar > View.

  • In Proxy Render Size, wonder why it is called Render, because you already rendered your proxy file.

  • It is set to Scene Render Size, so the proxy currently have the same size as the output.

  • Remember what proxy resolution you rendered at.

  • Set Proxy Render Size to the percentage of you build proxy.

  • Wonder if the preview actually is using the file or is just down scaling the original file on the fly. (If the checkbox in Strip Proxy & Timecode is unchecked the proxy file will not be used, but the image will still be downscaled).

  • Realize that only with 25% files you’ll get decent build times and playback, while the image looks really bad.

  • Wonder how you get it to render the full selection(by using Proxy Settings > Set Selected Strip Proxies and Rebuild Proxy and Timecode indices).

  • Wonder if you can batch convert files to proxy without adding them to the Sequencer(you can’t).

The current UI:
image image

The Optimal Solution:
(not feasible with the current manpower):
Would be to make everything automated, so all you would have to do was to set in Preview if you want to see proxies or not, and then all proxy health checking and building would happen fast and unnoticeable in the background, however the proxy building is slow(single threaded? color management?) And auto-checking health might be slowing down Blender and may be difficult to make fail safe in all situations.

Alternate Solution: Improve UX solution and Keep Current Functions:
(feasible with the current manpower):

  • As it is confusing what settings belong to either active strip and selection these things needs to be separated.
  • As it is confusing to set one or more proxy resolutions before building and remember the build values in order to set them in the Preview sidebar, this needs to be simplified.

In order to simplify it the general proxy settings needs to be moved into the User Preferences:


Maybe Overwrite should be added here too?
On the File Format option, the current format has okay playback, but do lack elements.

  • For equal playback but smaller files h.264 can be used.
  • For using opacity png in a Quicktime container can be used.
  • For high quality and good playback(but large files)DNxHD can be used(source needs to match the specifications for resolution), also supports higher bitrates(10+).
  • For high quality and high bitrates(10+) EXR should be used. For colorgrading on proxies.

On the Resolution, many codecs needs resolutions in fixed multipliers ex. x16, x8 or x2 to work.

On Quality, that is the bitrate.

On Timecode, some codecs do not support embedded timecodes(timecode usage is completely missing everywhere else in Blender).

For (Re)Build of the proxy of the current strip the options in the Strip tab could be limited to this:
image
A ‘Delete Proxy’ button could be added here too.
This panel should only work on the active strip, unless +Alt is pressed, in which case it should operate on the full selection.

For enabling and building proxies for the full selection that should be done through the menu:


Delete Proxies should be added to this menu.

The Preview Proxy panel can then be as simple as this:
image

Finally, proxy generation on import(using the User Preferences settings)could be added to the File Browser sidebar(as Eithan is working on: https://developer.blender.org/P1280 )


This option should also be exposed in the “Adjust Last Operation” panel, since the VSE workspace File Browser has no sidebar.
This panel could also have a Batch Proxy Build of Selected button, which builds proxies for all selected files, but do not import them into the Sequencer.

2 Likes
  • Wonder why you would need Timecode?

Frame-accurate seeking and processing requires correct timecode information, which not all video files provide themselves, see here

  • Wonder why you would need JPEG quality, when the proxies are video files?

I believe the proxy video files use the MJPEG codec, which is literally a sequence of compressed JPEG images. Hence the quality control setting.

  • Wait(no mult-ithreaded processing on the Blender side)

Multi-threading is definitely being used in Blender, but it will not matter for generating a video proxy. The whole file needs to be processed first before you can use it as a proxy. Plus, a single video file is produced for each proxy, which can only be encoded linearly.

By the way, I agree that the current UI and workflow for proxies isn’t ideal :slight_smile:

1 Like

@tintwotin you read my mind about some of the confusion first encountered when trying to use proxies. I use them in Resolve with no problem but Blender always makes me work harder for a similar result than I feel is needed. Some of your "thinning " down suggestions and moving to User Prefs makes sense. At the moment the lack of timecodes and integer frame positioning for audio mean I’m moving a lot of stuff back to Resolve for final processing. Also, some of my source video 4K, 10bit, All-I, chokes in Blender (and nearly every other package) but being able to create a proxy quickly would give me more reason to use the VSE. Similarly with Prores. Is there any point in letting the user choose whether the proxy is “optimised” for playback or editing+playback as different codecs might perform better in each instance. Also, a Delete Proxies for the whole project would be helpful for after a project has completed.

1 Like

I’m pro this. The UX is really bureaucratic, and yes, you need to make this automated. A proxy is a system to make editing responsive and quick, and right now the method to implement proxies outweighs the gains and makes the whole VSE editing experience ultimately sluggish, hard to learn and slow. It would be wonderful to implement your proposals…

2 Likes

Is there consensus of the first step to move the proxy render settings(Resolution, Quality, Timecode Index, Custom Directory, Overwrite(?)) into the User Preferences(and make them system settings) and and clean up the Sequencer and Preview Proxy panels accordingly?

What do @billrey @iss say?

In few words I am fine with making automated and simplified process.

I would expect to:
Go to user preferences, enable automatic building (at least it must be possible to be disabled).
Choose my favourite size and quality (we can have sane defaults).
Then any new project will have appropriate “render size” on load.
And any added strip will be set up to use proxies per user settings and automatically added to proxy builder queue.

Build proxy in file browser sidebar seems a bit redundant, but why not I guess.
Same goes for button in strip tab, it was moved to another tab, because you press it only once and you don’t need to see it again.

Adding new codecs and adding support for arbitrary resolution is quite out of scope of usability discussion. Timecodes are currently not quite implemented, so they can (should IMO) be hidden.

This is really on UI and file browser team to decide…

Not really, what if you have 25% and 75% sizes built, and you need to switch between them often?

I am not sure, if we hould have such fancy system, if it should apply to movie clip editor as well.
I would imagine that its users are likely to use only cache, but I don’t really know.

So you would go for a fully automatic proxy build of several optional resolutions and an option to select them in the preview.

If you prefer automatic(and the coding challenges that comes with it), it would be even more optimized if there was only one (value slider)resolution and the Proxy On/Off button the User Preferences would not only determine if the proxies are build automatic, but also could set the Preview to run in Proxy mode.

So in other words there would only need to be proxy related UI in the User Preferences and the default setting is On(afaik this is the way it is in Olive now). So users wouldn’t even notice or think about that they’re editing proxies, but just enjoy working in the VSE with good performance and export in high quality.

This is how it works now.

I am not exactly sure why we can have multiple sizes. Perhaps nobody builds more than 1.

I don’t really see benefit of greater granularity of sizes than we have
If you want to change this, you have to be prepared to make changes in proxy system and all editors that uses it. I think it would be a lot of work for very little benefit.

This is not really desirable, I edited 4K video, at 25% proxy size and use 240p high speed footage in one project. You just don’t want any proxy on that 240p clip or it will be unusable.

That’s why I am saying, have sane defaults, make user take only one action - click on “auto use and build proxy” checkbox.

You should always be very careful with decisions to limit existing functionality. You will likely break workflows and old files.

Well, then we’re back at this solution from above:

Which basically is moving proxy build settings from Strip into System, so they can be exposed in the User Preferences. And the other panels can be cleaned up.

Why make the proxy system-wide preferences? Why not make them a per-project setting like the render output resolution? When editing 1080p footage proxies at 50% might be fast enough, while for 4k proxies at 25% is probably what you want.

One thing I keep hitting upon in 2.8 compared to 2.7 is that the proxy settings are now on a different tab, and so I find myself constantly having to switch between tabs (which is less efficient than scrolling in this case).

Another reason for allowing a bit more control over proxy size per strip: the compression of the cached proxy files isn’t very good, so for large source videos the proxies can easily end up being in the same range of file size as the original input files. Which can be another factor in deciding what proxy size one wants to use.

Dealing with proxies currently is laborious and confusing. It could be so much simpler.

I think yes, it makes sense choose the proxy settings in Preferences, and to just have a global Proxy/Original toggle for the entire sequence.

@billrey As it is not possible with the current VSE manpower to implement a fully automatic and fast proxy solution, what are your thoughts on this mainly UI clean-up solution, which could be feasible to implement in the not too far away future?

1 Like

Are Proxies really a system preference? Would it not make more sense to have the proposed options reside in the view settings? That way they would get saved with the project and would still be in effect using different systems, User Preferences or even factory startup.

Richard is currently only endorsing presets for proxy settings for new strips, which means that this proposal to clean up the proxy UI will not be used.