VSE Strip retiming feedback

Feature & Design Feedback
1

The retiming feature has landed quite some time ago. There have been ongoing development coordinated mainly on gitea task While some polishing is still needed, I would like to gather feedback on how it can be further improved.
The feedback is collected in TODO task

Currently biggest outstanding workflow issues are:

  • Change key translation behavior, such that existing retiming is preserved (all remaining keys are moved)
  • Easier interface to input retimed segment speed
  • Better integration/workflow with freeze frames, so these could replace current Hard Split feature

To make retiming more versatile, reverse playback would be nice, but this is bit more challenging to implement for sound strips. Perhaps it could be implemented for video strips only in meantime.

As for glitches and incorrect behavior, please report these as bugs.

4 Likes
2

maybe freeze keys could be regular retiming keys with 0% speed and allow speed transitions to 0.

3

First of all thank you for doing it. As for feedback, I have just one big feedback: Bring back the tool.

Removal of tool was unnecessary and shouldn’t have happened before the user feedback. That was a wrong decision made without considering people who actually use VSE and Blender daily, by people who dont.

While I appreciate all the operators and toggles that we have now, I wouldn’t want them removed of course, I just realized when this thread opened that I haven’t actually used retiming tools after tool was removed. I switched back to DaVinci without even thinking about it. Because I really LOVED the tool, it was easiest retiming tool I’ve seen in video editing software. It was so intuitive, everything done with just mouse clicks. UI was very good, interactive, everything Blender should be. I tried retiming today and while its good, its not as good as tool.

I understand limitations of tools, mainly working on multiple strips at once, but since we now have operators that adress those issues, why not give us tool back? What was it hurting? Francesco replied to me on design task that tool didn’t fit into Blenders design, how so? That was never answered, and I dont think thats true at all, it feels like quick justification for something that was ultimately personal taste.

I really liked lines in the middle that you could drag. Double-clicking on text to change speed. Please, just give us tool back, it wasnt hurting nobody :frowning:

Smaller feedback:

image

That background line isn’t dark enough, its hard to see without thumbnails too but with thumbnails on its impossible to see, keyframes get lost. Would be nicer if it was made bit darker

1 Like
4

Some more things from stress-testing it:

  • Sometimes I get crash when deleting two keys at once, but sometimes not. Cant catch exactly when to file a bug report.

  • Deleting keys opens up this redo panel that says “Delete Strips” and has option to delete data. I’m assuming delete strips code is reused here, but this shouldn’t pop up.
    image

  • Once you add retiming keys you cant get rid of them on a strip. Its not a big problem since you can disable them and not touch accidentally, but it gets cluttery when you split the strip into multiple smaller ones, and all of them have start and end keys that do nothing. Maybe Reset retiming should also remove start and end keys? Or even better would be if that was the option in Redo panel, so you can choose if you want to keep them or not.
    Or maybe when you press Ctrl R if strip has only two retiming keys and 100% speed, hide keys and text entirely.
    image

  • Freeze Frame isnt pronounced enough. Breakdown keys are fine, but I think background color should change too, so that its more visible

  • Impossible to select first key if its right next to other strips last key. It always selects last key instead. No matter where you click.
    Video

  • R key to change speed becomes buggy and useless if you have retiming keys. Couple of issues that I detected:
    – When it pops up number written in it is of the last segment you retimed. So for example I have just one key in the middle, I select it, press R to retime it to 70%, next I select last key and press R, window pops up with 70% written in it, instead of 130% that it actually is. When I slide that number my retiming key jumps across to where it would be if it was actually 70%. Its very weird to describe, but basically it doesnt show you percentage that you actually have on selected key.
    Video
    – Number field doesnt have percentage subtype, it should show % sign at the end.
    – When you dont have any keys selected and press R it starts retiming last key. Its expecte when you have only start and end keys, but when you have multiple its confusing. I think if no keys are selected operator should tell user that its retiming last key. Perhaps change the name of pop-up from “Set Speed” to “Set Speed for Last Retiming Key”. That clarity is needed for good UX I think.
    – If you have Freeze Frame use Set Speed it gets very confused. Keys jump around, suddenly number is in decimals between 0 and 1. If I use retiming on Freeze Frame key and type 0 it jumps right next to previous key, and if I type any other number it still jumps there? So obviously you cant retime freeze frame keys, so I think it should be disallowed to do so and mess up your retiming like that. I think if freeze frame key is selected operator should fail and report pop up in status bar that tells you why.
    – Same thing is happening on speed transitions, but I’m big confused about them now so I cant go into details.

5

A user video on Retime:

1 Like
6

They technically are normal keys. Transition to 0 is possible technically, will have to do some testing and bit of code refactoring, because mixing keys is already pretty complicated.

@nickberckley I am not sure if tool can still be used along with current implementation, but I don’t think it would make sense to re-implement it

I have fixed crashing when deleting keys, the other issues I should fix soon. Some though are bit complicated - like using R key, when there are keys already. I should limit operator to work only on strips with no keys inside.

Reset retiming operator does remove all keys, but “fake keys” are visible, since retiming keys visibility is enabled. I would say, that it should also disable keys visibility.
Most of issues you mentioned here are quite valid bug reports, but it is true that it is quite a bit of them, so I will make multi report on tracker and reference that in commits.

And since I am here, I will add feedback too - Make transition/freeze creation modal, instead of using hardcoded length.

7

thank you @iss for the work, i tested the Strip retiming and this is my feedback:

  • adding just one additional key ( so we have 3 keys and two speed segments ) it works well also visually dragging the mouse: i can move the keys and the strips dimension and relative frames change autimatically and correctly !
  • when adding more than one key i start to have troubles: for example if i add 2 additional keys ( so we have 4 keys and three speed segments ) i cannot change the middle speed segment without effecting the other speed segments.It would be nice an option like ( CTRL or ALT key ? ) to increase or decrese the strip dimension/frames when dragging the keys that are not the first and the last keys.
    )
  • Editing the speed “R” seems to not work well ( i select the keys and the value do not correspond to the speed segment selected ). “set speed” should change the segment speed for what i understand.
  • preview during transform seems to not work and it could be useful when moving the retiming keys to understand what keyframe we are moving.
    In fact when i move a key retiming if the playhead is in the active strip the preview changes could be confusing.
  • the keyframe are not exposed in the graph editor in particular for the speed transition control like in the “speed control effect strip” for a more controlled speed ramp effect.
  • for the “slow motion” i do not understand if a frame blending interpolation could be applied like for the “speed control effect strip” option.
  • i noticed in the FFMPEG the option to use the “minterpolate” filter with motion interpolation modes. Could be implemented maybe in the future as an option?

https://ffmpeg.org//ffmpeg-filters.html#minterpolate

What about the “speed control effect strip” ?
Do you plan to integrate it to the “strip retiming” or to work side by side like two options for retiming ?
the pro in my opion for now are:

  • we can animate the property of “multiply” “leght” “frame number” that are exposed in the graph editor.
  • the option of frame interpolation ( even if limited to crossfade blend ).

the cons are:

  • the strip dimension do not change automatically
  • the retiming keys are not visible in the VSE timeline strip

Anyway thank you for the work, it is very promising !

8

@Andrea_Monzini Thanks for feedback, and sorry, that I did not get to this earlier. I will add these points to TODO task, and will respond to some points here.

Right now, I have implemented only ctrl+click to select key + all remaining. But I agree, that this is good idea. There are arguments, that what you propose should be default behavior.

Retiming keys are not keyframes. Keyframe represent a value at specific frame. Retiming key represents a frame itself. You could argue, that it could be shown there, which is true, but the range would have to be safeguarded and editing would feel weird due to strips possibly moving around when moving keys in different editor. Also there would be no modifiers etc. It would be quite lot of work to do this and it will only result in hacked in functionality with lot of limitations.

Unfortunately not yet, but it is good idea, since it is simple. Slight challenge could be, that VSE uses simplistic frame remapping if movie FPS does not match scene. If this is combined with retiming it will have to be implemented on higher level to work correctly. So this would be some kind of inherent effect of strip like blending is. Even with speed effect the implementation may be not where you would be looking for it. It is not a problem, just thinking about this loudly.

On that note, the mentioned effect by ffmpeg would be great addition too. I think this would be possible to implement. It could be, that this is missing from our precompiled libraries. Will have to check

This effect is just bad from design perspective. Multiply is the same as setting strip speed, length is the same as moving last key, frame number is same as moving particular frame somewhere else. First 2 modes can’t be used reasonably with animation(I know, that one user finds this really useful for their workflow). Animating frame number may be actually faster workflow, than current retiming system provides. But I don’t see users having spreadsheet with how they want to remap frames when actually editing some footage. But if that is necessary, such workflow can still be done with python script.

1 Like
9

  • About the retiming keys do you plan to enable the “preview during transform” option?
    i think it could be useful in particular when i move a key retiming if the playhead is in the active strip the preview changes is confusing.

  • if not possible/complicated to expose the retiming keys in the graph editor what we can be done to control the speed transition with a custom speed ramp effect ?

thank you.

10

Yes, I have included link to TODO task in top comment.

Now I am not sure what do you mean by custom ramp. You can add multiple keys with different slopes and transitions between them. If you refer to the transition shape itself, you have 2 options pretty much - Use speed effect or add retiming keys for each frame for duration of this ramp.

Currently the ramp is defined as perfect circular arc between 2 linear segments. Not sure if this is the most ideal curve shape. In future you may be able to change curve shape. I have experimented a bit with long and short transition duration, and IMO different curve shapes would be barely perceptible. I would need to see samples where custom profile is needed to achieve specific effect.

1 Like
11

Basically i was asking if it is possible to change the acceleration curve ( or better the speed variations curve ) , maybe with just 3 presets like slow-in ( fast-out ), linear, fast-in (slow-out).

But perhaps in a 24 fps timeline using 2 or more “speed transitions” segments to approximate the curve would have a negligible difference.
Definitely it would be useful to prepare some samples.

As reference slowmoVideo ( free and open source project with “Optical Flow” slowmotion ) can use Bezier curves for speed variations:

Docs:
http://slowmovideo.granjow.net/docs.html

Repository:

Could be interesting to evaluate the possibility to use slowmoVideo as a library ( as alternative to the FFMPEG “minterpolate” ) :

slowmoVideo discussion about Create re-usable slow-motion library and Discussion: Future of slowmoVideo

There is also the Flatpak repository to test it :

Beside AppImage:

Thank you

1 Like
12

That looks quite nice. but author is suggesting, that this is not very suited for realtime playback. FFmpeg’s minterpolate option can exploit the fact, that codecs do store and use information about motion to reconstruct the image, so the decoder does not need to do much analysis in order to interpolate between frames. This should speed up the process significantly. But ultimately I did not do any benchmarking yet in this area.

You could use multiple libraries for playback/render though, since I can imagine minterpolate to be bit more glitchy. But again, this would require some investigation.

1 Like
13

I’m testing the retiming feature again and recollecting feedback and issues from myself. I’ll just paste it here to keep it around and maybe have a discussion about it.

I now notice that a lot has improved since last year! But there’s still plenty that’s not ideal.
Also it’s a bit too much to report as individual bugs right now.

Splitting Strips

Splitting a strip that already has retiming keys will lead to issues when editing the speed of the retiming keys afterwards.
When selecting and changing the speed of the automatically placed start/end retiming keys, it will not correctly take the visible strip range into account and mess up the timings.

Creating a hard split is even worse. This will completely break the retiming keys.

Synced Selection on Connected Strips

The automatically placed retiming keys at the end of the strip are not selected together on connected strips (typically video & audio strip).

The selection is also not synced when using Shift + Select on retiming keys.

Soft Transition

Soft transitions are not very well exposed. The operator to create them is missing from the I key menu.

Once they are created it’s also not possible to properly change the speed to the left of it. When creating the soft retiming keys and pressing R to set a speed, it will set the wrong speed.

Odd Situations with the Key Representation

I mentioned this before but it’s still very hard to select the last retiming key. On long strips it might require zooming/panning very far to see it. I still see this as an inherent flaw of the key representation instead of selecting the range inbetween …

Freeze Frame and Slide

When using ‘Add Freeze Frame and Slide’ it will not preserve the speed of the retiming key to the right of it.

Freeze frame and slide is also contextual to the selection. If a retiming key is selected it will create a freeze frame there. If just the strip is selected it will create the freeze frame at the current frame.
Maybe this should be two operators? Or at least mentioned in the tooltip. I find this confusing.

Similar issue. When multiple retiming keys are selected (which can be very common when using Ctrl+Select ) it will create a freeze frame at EVERY selected retiming key. I don’t think this behavior is ever useful.

If you add a freeze frame to the last automatically placed retiming key, it will shrink the entire strip to a single frame long.

Misc

I think there’s in general an issue with sub-frame values? Some speed values cannot be used by the VSE. I wonder if there are any thoughts or plans about that.

Audio strips are also not pitch corrected when the speed is changed.

The pre-fetched frame cache seems to easy become invalid if the retiming and cutting of strips is used. Using Ctrl E to reload the strips is still commonly needed.

On long strips the percentage value can get out of view. Should remain centered on visible strip

On a broader workflow level, I’m not sure how to use retiming keys in combination with other keyframe animation on strips. When changing the speed of a range of a strip, the keyframes within that range will not be adjusted.
And because retiming keys are not keyframes, they canot be manually edited together either.

4 Likes
14

Hi,
Any chance the python api could include retiming_keys for all strip types that support them?

In my situation, I have lots of retiming keys on lots of image sequence strips which I would like to replace by scene strips. Since the strip type is read-only in python, I can’t just change it and keep the retiming. However, if I could copy-paste all the retiming keys from one strip to the other, I could carry the retiming from the image sequences to the scene strips, but right now I can’t because only 'MOVIE' type strips have the retiming_keys property.

15

This is a bug really will look into fixing this. RNA really dislikes calling RNA_api_sequence_retiming_keys() multiple times, so I hope, that this isn’t a limitation that would be hacky to work around.

1 Like