[MPlayer-dev-eng] [PATCH] yuv 4:4:4 scaling bug

[Loren Merritt]

bug #1: swscaler from rgb to yuv 4:4:4 (or vice versa) goes through 4:2:2

mencoder in_rgb.avi -ovc raw -vf format=444p -o a_444.avi # line1
mencoder in_rgb.avi -ovc raw -vf format=422p -o b_422.avi
mencoder b_422.avi  -ovc raw -vf format=444p -o c_444.avi
# a_444.avi and c_444.avi are identical

line1 outputs,
  SwScaler: reducing / aligning filtersize 1 -> 4
  SwScaler: reducing / aligning filtersize 5 -> 4
  SwScaler: reducing / aligning filtersize 1 -> 1
  SwScaler: reducing / aligning filtersize 1 -> 1
  SwScaler: BICUBIC scaler, from 0x42475220 (BGR ) to 0x50343434 (P444) using MMX2
  SwScaler: using 4-tap MMX scaler for horizontal luminance scaling
  SwScaler: using 4-tap MMX scaler for horizontal chrominance scaling
  SwScaler: using 1-tap MMX "scaler" for vertical scaling (YV12 like)
  SwScaler: 640x360 -> 640x360
  SwScaler:Lum srcW=640 srcH=360 dstW=640 dstH=360 xInc=65536 yInc=65536
  SwScaler:Chr srcW=320 srcH=360 dstW=640 dstH=360 xInc=32768 yInc=65536

The attached patch fixes this. Since swscaler provides flags to disable 
this behavior, I just used them. But is there really any situation in 
which you would want intermediate downsampling? If not, it would be better 
to modify swscaler instead.


bug #2: swscaler from 4:4:4 to any other format produces chroma artifacts. 
The artifacts are independent of what the destination format is.
The above patch fixes this for 4:4:4->rgb, but not to subsampled yuv 
formats. I don't know where to look for the cause of this.

mplayer in_rgb.avi -vo png -vf format=yv12,scale
  produces http://students.washington.edu/lorenm/ed_good.png

mplayer a_444.avi -vo png -vf format=yv12,scale
  produces http://students.washington.edu/lorenm/ed_bad.png

--Loren Merritt
-------------- next part --------------
Index: libmpcodecs/vf_scale.c
===================================================================
--- libmpcodecs/vf_scale.c	(revision 19447)
+++ libmpcodecs/vf_scale.c	(working copy)
@@ -221,6 +221,8 @@
     sws_getFlagsAndFilterFromCmdLine(&int_sws_flags, &srcFilter, &dstFilter);
     int_sws_flags|= vf->priv->v_chr_drop << SWS_SRC_V_CHR_DROP_SHIFT;
     int_sws_flags|= vf->priv->accurate_rnd * SWS_ACCURATE_RND;
+    if(best==IMGFMT_444P || outfmt==IMGFMT_444P)
+        int_sws_flags|= SWS_FULL_CHR_H_INP|SWS_FULL_CHR_H_INT;
     vf->priv->ctx=sws_getContext(width, height >> vf->priv->interlaced,
 	    outfmt,
 		  vf->priv->w, vf->priv->h >> vf->priv->interlaced,