[FFmpeg-devel] Mixture of homographies video stabilization

[Matthew Lai]

Ah cool! I didn't see that before. I have only recently started looking
into homographies, and most papers use SVD.

Good to know re. ME. I should stop trying to guess where bottlenecks are
likely to be without profiling :).

On Tue, Aug 7, 2018 at 12:33 AM Michael Niedermayer <michael at niedermayer.cc>
wrote:

> On Mon, Aug 06, 2018 at 03:49:00PM +0100, Matthew Lai wrote:
> > Sorry the matrices are actually 3x3, and most of the time will be spent
> on
> > solving a 8x9 * 9x1 = 0 systems. And then all the feature points will
> have
> > to be multiplied by the 9x1 matrix reshaped into 3x3.
> >
> > The math is here:
> >
> https://www.uio.no/studier/emner/matnat/its/UNIK4690/v16/forelesninger/lecture_4_3-estimating-homographies-from-feature-correspondences.pdf
> >
>
> > Multiplying 3x3 matrices is trivial, but SVD isn't. SVD is the standard
> > approach to decompose a homography matrix. I have not looked into
> > alternative approaches.
>
> Iam a bit puzzled.
> IIUC SVD is the slowest part of the filter from what you write above and we
> have no evidence that it is needed. And you did not look at alternatives ?
> ;)
>
> Quoting "Calibration-Free Rolling Shutter Removal" from
>    Matthias Grundmann1,2           Vivek Kwatra1      Daniel Castro2
>    Irfan Essa1,2
> ...
>     Aggregating all linear constraints Ax for each feature match
>     (x, y) yields an homogenous linear system, which can be
>     solved for under the constraint ||h||2 = 1 using the SVD
>     of A. Alternatively, the system can be transformed into a
>     homogenous system by explicitly setting the bottom right
>     element of each homography to 1, i.e. hk (3, 3) = 1 for all k,
>     which is a reasonable choice for video, as the small inter-
>     frame motions are virtually free of degenerated cases.
>
> this sounds like theres a very easy way to avoid
> SVD.
> from what i remember the main point of
> SVD is its ability to work with unstable / degenerate cases.
> But, in which case is such a degenrate case a good estimation for the
> motion/transform ?
>
>
> >
> > Cool! Will see re. ME. It's likely to be the performance-limiting part,
> and
> > it would be nice to run that on GPU.
>
> just a quick reference point, ME for encoding is many times real time.
> for example a 720x576 video encodes here with 1227 fps on a many years old
> box. That includes alot of other things like decoding and all the bitstream
> encoding and transforms, not just ME. And this is 100% on the CPU
>
> [...]
> --
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> I have never wished to cater to the crowd; for what I know they do not
> approve, and what they approve I do not know. -- Epicurus
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>