[FFmpeg-devel] Mixture of homographies video stabilization

[Matthew Lai]

Ah ok thanks! I'm surprised no one has need a linear algebra library. I
guess there's OpenCV and people use it to do the heavy lifting?

Will look into the API more.

Thanks
matthew

On Thu, Aug 2, 2018 at 3:31 PM Paul B Mahol <onemda at gmail.com> wrote:

> On 8/2/18, Matthew Lai <matthewlai-at-google.com at ffmpeg.org> wrote:
> > Hello!
> >
> > I want to write a more advanced video stabilizer for libavfilter (*),
> > implementing the algorithm described here -
> >
> https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/37744.pdf
> > The focus of the paper is rolling shutter removal, but it builds on top
> of
> > another algorithm that does full frame stabilization, and the new
> algorithm
> > does that as well.
> >
> > This is the algorithm used in YouTube's stabilizing filter, and is state
> of
> > the art. Adobe calls it Warp Stabilizer (it's the same thing as far as I
> > can tell from public information anyways).
> >
> > 3 questions:
> > 1. Is there a linear algebra library already in use? I didn't see
> anything
> > in configure, but would be surprised if none of the existing filters work
> > with matrices?
>
> There is no such library here used. There are indeed video/audio
> filters that work with matrices.
>
> > 2. Is there anything to watch out for re. a high frame delay (say a few
> > hundred frames)? Looking at the API, I don't see a callback to flush out
> > remaining frames when input frames are finished? Is doing it in two
> passes
> > the only option?
>
> It is handled internally, there are two internal APIs, activate one and
> legacy.
> With legacy you can flush frames when you receive last frame from input.
> With newer, activate API, its similar.
>
> > 3. doc/writing_filters.txt says only slice threading is available. That's
> > not really possible with this filter, but frame threading is. Can I just
> > buffer frames internally (which I need to do anyways to smooth out
> motion),
> > and do my own threading?
>
> You could do it.
>
> >
> > * vid.stab is good for what it does, but it only does rotation and
> > translation, and doesn't handle zoom, perspective distortion, or rolling
> > shutter. This means it's limited when it comes to things like scuba
> diving
> > videos, where the camera is filming the seabed at a small distance and at
> > an angle.
> >
> > Thanks!
> > Matthew
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel at ffmpeg.org
> > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>