VFX: Global Camera Solver Feedback

This is a new camera solver based on the approach described in Global Structure-from-Motion Revisited (GLOMAP).

It is an alternative to the existing incremental and tripod solvers. Instead of adding cameras frame-by-frame, it optimizes the rotations and positions of all the cameras and tracks in one large Ceres problem.

The “Tripod” checkbox in the Solve panel was replaced with a dropdown with 3 options:

  • Incremental (the current solver)
  • Tripod
  • Global (this new solver)

Usage:

  • Open a video in the clip editor
  • Track 2D features
  • Switch to the Solve tab and select “Global” in the Solver dropdown
  • Solve Camera Motion

It’d be great to get some feedback on what scenarios this performs better/worse than the existing incremental solver.

Build Download (PR)

11 Likes

My first tests in simple shots were succesful. The result was accurate and in ways improved to the Incremental Solver. (resulting overall initial scale for example)

I will do some more tests in more difficult shots.

3 Likes

I tried the new solver in some very difficult shots as well. Mostly green screen shots with minimal markers or environment features. This resulted in jitttery camera animation without any movement coherence. The native tracker didn’t give anything useful either. So the Global Solver still needs good tracking data to return good results. It will not be a magic bullet to solve probelamtic shots.

To be fair those were shots that needed a lot of manual work and corrections to assist in solving the movement. It is not a problem of the solver. Only Syntheyes managed to solve them fast when we were working on them.

I know this is not the topic of this thread but maybe the tracking worflow/tools need some love as well.

4 Likes

Hard shot, 4k sony HLG with some motion blur: 1.07 with the Incremental solver - A billion px error with Global solver.

I’m not a programmer or expert but I feel that compared to other software, the 3d tracking workflow use old tech. The main bottleneck it’s in the track method, if tracker “jump” then the solve is hard.

Davinci and others tracker, for example, use the optical flow method while some plugin embraced OpenCV or photogrammetry stuff like Colmap CLI.

2 Likes