[Ffmpeg-devel] Snow motion blocks

[Yartrebo]

On Fri, 2005-04-15 at 12:49 +0200, Michael Niedermayer wrote:
> Hi
> 
> On Friday 15 April 2005 07:09, Yartrebo wrote:
> > I've been playing around with the motion block functions and I seem to
> > be getting consistently better results using much simple functions than
> > the one currently in use (int am= 20*(a2+a3) - 5*(a1+a4) + (a0+a5);).
> >
> > First off, vs. dsputil, complex is inferior and simple is superior, but
> > I need to run more varied tests (I didn't test that relation as much as
> > simples vs. complex).
> >
> > For these tests, I've
> >
> > I've been playing with the following three functions for mc_block:
> > 1 - complex - int am= 20*(a2+a3) - 5*(a1+a4) + (a0+a5);
> > 2 - middle - int am= 18*(a2+a3) - 2*(a1+a4);
> > 3 - simple - int am = 16*(a2+a3);
> >
> > In lossless mode, simple + hpel seems to give the best results. For
> > lossy mode, simple + qpel seems to give the best results.
> >
> > sample: doodli-do (cartoon music video - raw tv cap)
> > quant:2
> > simple + qpel: frame= 2698 q=0.0 LPSNR=Y:41.18 U:44.76 V:46.04 *:42.19
> > size=   25482kB time=90.0 bitrate=2319.0kbits/s
> > complex + qpel: frame= 2698 q=0.0 LPSNR=Y:41.11 U:44.67 V:45.96 *:42.11
> > size=   26928kB time=90.0 bitrate=2450.5kbits/s
> >
> > sample: charlie chaplin (black and white video - mpeg4)
> > quant:lossless
> > 8809128 Apr 14 23:09 spaceballs_mc_complex.avi
> > 8720006 Apr 14 23:26 spaceballs_mc_complex_qpel.avi
> > 8843648 Apr 14 23:28 spaceballs_mc_full_complex_qpel.avi
> > 8384154 Apr 14 23:40 spaceballs_mc_full_simple.avi
> > 8623338 Apr 14 23:20 spaceballs_mc_full_simple_qpel.avi
> > 9032272 Apr 14 23:37 spaceballs_mc_full_trivial2.avi
> > 9006404 Apr 14 23:31 spaceballs_mc_full_trivial.avi
> > 8796270 Apr 14 23:05 spaceballs_mc_mid.avi
> > 8771816 Apr 14 23:01 spaceballs_mc_simple.avi
> > 8700496 Apr 14 23:22 spaceballs_mc_simple_qpel.avi
> >
> > Full refers to all motion blocks being handled by mc_block, and the
> > dsputil code is not called. Otherwise the motion blocks are handled by
> > the normal functions.
> >
> > sample: Jakers (CG animation - mpeg4)
> > 1558370 Apr 15 00:10 jakers_full_complex_quant2.avi
> > 1509116 Apr 15 00:07 jakers_full_complex_quant2_qpel.avi
> > 16794250 Apr 14 23:54 jakers_full_simple_lossless.avi
> > 1480450 Apr 15 00:01 jakers_full_simple_quant2.avi
> > 1477526 Apr 15 00:03 jakers_full_simple_quant2_qpel.avi
> >
> > for the quant2 clips, psnr is better for the smaller (better compressed)
> > files, but I forgot to write down the exact psnr values. They didn't
> > differ by more than .10, and favored simple and qpel over complex and
> > hpel.
> >
> > I'll be running complex+qpel and simple+qpel on a full-length movie
> > (Matilda) overnight. Unfortunately, it is MPEG4, but I have no live-
> > action material that isn't divx or xvid, so it'll have to do for now.
> >
> >
> > Would you have any explanation for the empirical results I'm getting?
> > Even your commented-out mid function (int am= 18*(a2+a3) - 2*(a1+a4);)
> > performed better than the complex one.
> 
> yes, i have
> if your source material uses bilinear (=simple) hpel then recompressing it 
> with anything else will lead to a quality loss, which might be larger or 
> smaller then the gain from the 6tap qpel filter, thus recompressing 
> mpeg4->snow is not a good way to compare the MC filters
> after all the goal is to be close to the original raw video and not to be as 
> close as possible to the artifacts from compression
> a similar situation would be trying to compress lowbitrate blocky video, 
> wavelets would always suck for this hugely compared to block based coders
> 
> one situation where bilinear might be supperior is very noisy video or anime
> 
> 
> >
> > Considering that simple is far faster to implement than complex, it
> > really is worth a deeper look (motion comp is a very large part of snow
> > decode time, especially with all my asm code applied).
> 
> yes, if you can find a simpler filter which works well compressing raw video 
> or very high bitrate mpeg2 then we certainly should do something here
> btw, tv captures might have been mpeg2 too, iam not sure how relevant that is 
> though
> btw, see http://meru.cecs.missouri.edu/free_download/videos/ (or try google, 
> like i just did after realizing all my raw video sequence bookmarks are dead)
> or https://www.ldv.ei.tum.de/liquid.php?page=70
> for raw video sequences
> 
> btw3, the current "complex" 6tap filter is from h.264 and they extensively 
> tested that its a good choice AFAIK
> 
> [...]
Thanks for the explanation. All my test samples fit one of the above
cases (either animated or transcoded from block-based - my tv captures
are raw, but they are animated).

I will try to find something that can implemented fast and give as good
or nearly as good quality. The current filter is the slowest part of
snow (with horizontal compose being the second slowest). In particular,
edge motion-blocks (mc_block) alone takes about 10-15% of decode time
with my asm code is use). Even if we only change the algorithm for edge
blocks, the change in speed could be quite dramatic.

Robert