[FFmpeg-devel] [PATCH] odd-height RGB->yuv420 fix invalid reads

[Reimar Döffinger]

On Fri, Oct 08, 2010 at 07:49:35AM +0200, Reimar D?ffinger wrote:
> On Fri, Oct 08, 2010 at 04:03:36AM +0200, Michael Niedermayer wrote:
> > On Thu, Oct 07, 2010 at 09:35:37PM +0200, Reimar D?ffinger wrote:
> > > Hello,
> > > while it will leave the last line(s) uninitialized (do they even exist/will
> > > they have been allocated?), below patch fixes the out-of-bounds reads.
> > > Should fix the crash part of issue 2279.
> > 
> > they must have been allocated or there is a bug in the allocation
> > and they should be filled
> 
> I'm not sure I understand really, but I think attached patch will
> fill them (somewhat) correctly.
> Note that the same issue (the uninitialized, not the crash part)
> still exists for odd width AFAICT.
> However for the given sample it at least solves all valgrind issues.

I think I can't think straight today.
Below patch is much simpler, however it does not initialize any additional
luma line the destination might have - this could be solved with a memcpy,
but then we should document somewhere that a odd-height 4:2:0 actually
must have that "additional" luma line.
Index: libswscale/rgb2rgb_template.c
===================================================================
--- libswscale/rgb2rgb_template.c       (revision 32448)
+++ libswscale/rgb2rgb_template.c       (working copy)
@@ -2263,6 +2263,8 @@
         }
         ydst += lumStride;
         src  += srcStride;
+        if (y == height - 1)
+            break;
 
         for (i=0; i<chromWidth; i++) {
             unsigned int b = src[6*i+0];