2D transfom processing, concatenation of transforms in VSE

[Prag]

I think 2D transform processing in VSE has some kind of a design issues. In general as far as I understand the image (and movie) strip processing is evaluated on every stirp and that causes:

1. the image is cropped based on the previous strip process.
2. the image is filtered on every separated strip process.

Example for scenario 1.
I load an image which is bigger than the format size (format is 19201080 and the image is 20482048). So it is obviously cropped to the format size:

The image can be transformed anywhere and it will be cropped to the format according to.

But if I apply a transform effect strip for whatever reason the image remain cropped based on the previous state:

The VSE scene seems like this:

And there is no option to disable this behaviour.

Example for scenario 2.
Same image is scaled down to 0.2.

Then a transform effect strip is applied with scale up to 17. There result seems like this:

But (obviously) if don’t use an extra transform effect strip instead I concatenate the two transform values (scale 0.5; scale 17 = scale 8.5) the result seems like this:

But we can’t ban transform effect strips. There are a lot of situation when they are needed.
The same issues are with the meta strips, because they are also introduce a new 2D image processing.

There is a good description of this problem in connection with the compositing software Nuke:
Concatenation of transforms in Nuke

So as far as I see the 2D transformation processing (in VSE) should be concatenated and the process flow should be something like this:

{img transform (image strip for eg.) → image transform (effect strip for eg.) → image transform (meta strip for eg. ) → etc.} → transform concatenation → evaluate the bounding box of the image (with an option to keep the whole image outside of the format or crop the pixels outside) → crop the image base on the scene format → filtering

That kind of design would help to keep the original image quality as much as possible but also would help to decrease the cost image processing.

What do you think?

[Bachadam]

I think you should signal it as a bug or a limitation of image transformation.

The compositor transform Node has not this behavior. So I think it could be fixed.

Edit:
But another problem is that, what if user expect the first transform image to be crop before the second transformation.

• To me the best solution to handle this is to create a new VSE 2DTransform modifier.
  So we could transform image in no destructive way. When that modifier is applied then the image transformed will be crop.

[tintwotin]

@Prag The Transform Effect Strip is in the process of being removed: ⚙ D13563 VSE: Remove transform effect strip

[iss]

In theory this would be possible to implement, but with VSE this would require significant changes in how images are rendered. It is much simpler for node systems to concatenate transforms, but VSE does not use nodes.

Without nodes, transform effect would have to pass it’s matrix to input and there it would have to be decided whether it is possible to concatenate or not and this would have to be “signalled” back to effect strip. It’s possible to implement this relatively easily, this “signalling” can be used at other places as well, but code complexity would undeniably be increased.

Similar case of concatenation already happens with proxies and it really isn’t nice thing to work around. It has improved performance, but it has caused number of bugs as well. So I would say that this is mainly question of cost vs benefit vs priority. It would be best if you can provide workflows where this is actually an issue so there is more context to this problem. I could create TODO task for this.

In my opinion, modifiers would not help here too much, form implementation point of view they work same way as effects do. Also to not increase code complexity I would argue that node system should be implemented instead of adding more and more exceptions. I don’t think it’s that hard to do (not talking about node editor, just how code does actual processing), but obviously would take quite some time to define actual nodes, write code to construct, simplify and evaluate the tree.

[iss]

Just to clarify, I don’t think transform effect will be removed as it seems to be far more practical to keep it, but to use new transform properties only. This way old files will be unaffected and as it’s said here it’s still needed sometimes. My idea was that other effect type could be used as dummy type since all strips have transform properties, but I don’t think that was very good idea…

[Prag]

I’m not familiar with the code of the VSE but for me it sounds like a plan to implement a node system. I guess it should be a strategic decision to go that direction.

I created an example file to show 3 scenarios of the “same” transformations on multiple image strips.

1. Using meta strip (and transform effect stript) would be the obvious solution to transform logically connected / “hierarchical” image strips. But there is that concatenation problem.
2. Using only image strips (no meta strip) can be kind of a solution. But is not user friendly at all. It is more work to achive the “same” thing with meta strips and it is very hard to modify things after the image strips are animated. We have to key every image strip instead of one meta strip. AND the transformation is not filtered (related topics is here).
3. Using image strips and transform effect strips (no meta strip) basically the same issue as at 1. There is no transform concatenation.

I hope it helps.

Side note: unfortunatelly File / External Data / Pack Resources does not work with VSE.