[MPlayer-users] [BUG] swscaler conversion of yv12 to nv12 broken

[Guennadi Liakhovetski]

On Thu, 30 Jul 2009, Guennadi Liakhovetski wrote:

> Hi
> 
> I'm using mplayer on a SuperH platform with the sh_veu vidix driver, which 
> takes nv12 as an input format. It used to work at least with a version 
> from August 2008, a recent (21.07) SVN snapshot doesn't work any more. 
> Doesn't work means, that with the clean SVN version the image appears 
> black-and-white on the LCD.

Ok, problem fixed. Or more precisely, it's cause found and a crude 
solution produced, now it's up to you to produce a correct one. So, the 
cause is, that sws_getContext() is now picking up a wrong converter for 
the yv12 -> nv12 case. It first gets the right one - 
PlanarToNV12Wrapper(), but then it goes on and overwrites its first choice 
with planarCopy(). The reason, I think, is too broad criteria for the 
latter one. I narrowed them down with

Index: libswscale/swscale.c
===================================================================
--- libswscale/swscale.c	(revision 29435)
+++ libswscale/swscale.c	(working copy)
@@ -2791,7 +2791,9 @@
             || (isGray(dstFormat) && isGray(srcFormat))
             || (isPlanarYUV(srcFormat) && isPlanarYUV(dstFormat)
                 && c->chrDstHSubSample == c->chrSrcHSubSample
-                && c->chrDstVSubSample == c->chrSrcVSubSample))
+                && c->chrDstVSubSample == c->chrSrcVSubSample
+		&& dstFormat != PIX_FMT_NV12 && dstFormat != PIX_FMT_NV21
+		&& srcFormat != PIX_FMT_NV12 && srcFormat != PIX_FMT_NV21))
         {
             if (isPacked(c->srcFormat))
                 c->swScale= packedCopy;

As I said, this is, probably, not the right solution, but it fixes the 
problem. In the working version the test was just

        if (  srcFormat == dstFormat
            || (isPlanarYUV(srcFormat) && isGray(dstFormat))
            || (isPlanarYUV(dstFormat) && isGray(srcFormat)))

Looking forward to a correct solution of the problem.

Thanks
Guennadi
---
Guennadi Liakhovetski, Ph.D.
Freelance Open-Source Software Developer
http://www.open-technology.de/