[FFmpeg-devel] [GSoC] Motion Interpolation

[Michael Niedermayer]

On Mon, Jul 25, 2016 at 12:25:07PM -0400, Ronald S. Bultje wrote:
> Hi,
> 
> On Mon, Jul 25, 2016 at 5:39 AM, Michael Niedermayer <michael at niedermayer.cc
> > wrote:
> 
> > On Mon, Jul 25, 2016 at 04:05:54AM +0000, Davinder Singh wrote:
> > > https://github.com/dsmudhar/FFmpeg/commits/dev
> 
> 
> So, correct me if I'm wrong, but it seems the complete ME code currently
> lives inside the filter. I wonder if that is the best way forward. I
> thought the idea was to split out the ME code into its own module and share
> it between various filters and the relevant encoders without a strict
> dependency on avfilter/avcodec, or more specifically, AVCodecContext or
> anything like that?

there is little overlap between ME used in encoders and ME used
for finding true motion
These are different problems unless one considers just some
initial motion estimation pass that just serves as a start point
(that initial start pass code could be shared in principle ...)

For encoding you want to minimize the distortion after the decoder
and the bitrate, and you are restricted by the structure of the
codec like 4x4, 8x8 and 16x16 blocks with 1/4 translational motion
for example


For finding true motion for the purpose of filtering there is no
bitrate, there is no restrictions on the segmentation, precission,
type of motion or the number of motion vectors per pixel. distortion
does not even matter all that much, the vectors must match the
true motion of the objects and if you cant achive that making the
vector field smooth temporally and spatially is much more important
because having all vectors wrong in a smooth way looks much better
than having a tiny number of vectors wrong even if these match very
well the pixels
simple frame duplication is basically all (0,0) vectors from the
past frame and that looks ok


just think about what happens if you interpolate with all prefectly
optimal vectors per pixel but one single 16x16 block being entirely
wrong. Such a frame would have a randomly displaced 16x16 block in each
frame thats much worse than all (0,0) vectors
also that bad block could be 100% matching pixel wise between
frame 1 and 3 and still be totally wrong at the interpolated
position in frame 2 (which we create)
this is the big issue i had with mcfps, it all looks good except the
occasional vectors which match ok pixel wise but are just really
not right vector wise, like someones finger moving to his nose
because that matches well pixel wise ...
(it wasnt finger and nose but random blocks moving around very
 differntly from the actual motion which looks really bad)

Also another issue with code sharing ATM is that it would make
tuning the code harder, the ME code used for the encoders is quite
restricted in what it supports and its optimized for that.
IIUC for the filtering Davinder is still testing and tuning things,
i think thats much easier without draging
"shared with encoders and highly optimized for that" code along

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

I have never wished to cater to the crowd; for what I know they do not
approve, and what they approve I do not know. -- Epicurus
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20160725/b4b71535/attachment.sig>