[FFmpeg-devel] [PATCH / RFC] Deshake / stabilize filter

[Alexander Strange]

On Apr 12, 2010, at 12:49 PM, Daniel G. Taylor wrote:

> Hey,
> 
> Attached is a filter I wrote over the weekend for fun to stabilize an input video by shifting frames horizontally and vertically. It's fast (uses MMX/SSE/etc) and fairly accurate.
> 
> I am just learning about block matching, motion searches, and various other concepts and this is an algorithm I came up with reading various things online and in some free papers. The algorithm is described in the comments of the patch in plain English. There are still a few things I'm confused about / working on but here are some quick results from my Vado HD pocket camera in a real world scenario:
> 
> http://programmer-art.org/dropbox/deshake_dog_orig.mp4
> http://programmer-art.org/dropbox/deshake_dog_32.mp4
> 
> And here is an unrealistic example to show the strengths and weaknesses of the filter in its current state (e.g. no rotation support):
> 
> http://programmer-art.org/dropbox/deshake_table_orig.mp4
> http://programemr-art.org/dropbox/deshake_table_32.mp4
> 
> The filter is licensed under the LGPL and I'd like to:
> 
>  1. Ask for comments, suggestions, etc to make it better
>  2. Get it into FFmpeg trunk

libavcodec already includes motion estimation code which should be useful here, so you don't have to reimplement it using only fullpel exhaustive search.
Unfortunately it's not exposed on its own, so the mplayer filters that use it (like vf_mcdeint) encode to Snow and then read the motion back out of that, which seems wrong. And some of the me methods are actually controlled by dia_size instead of me_method for some reason.

It would be nice if those were fixed so filters could share algorithms properly.

> I also found this while searching and realized it uses a very similar algorithm but does a full matrix transform and detects rotation which seems works really well but is significantly slower and GPL:
> 
> http://public.hronopik.de/vid.stab/features.php?lang=en

Where possible I'd like to see Avisynth filters ported rather than transcode, since I have a feeling that they work better due to more people testing them:
http://avisynth.org.ru/mvtools/mvtools2.html
http://avisynth.org.ru/depan/depan.html

I haven't used either of those, but the manuals might be interesting.