[FFmpeg-devel] [PATCH] odd-height RGB->yuv420 fix invalid reads

[Michael Niedermayer]

On Fri, Oct 08, 2010 at 07:57:28AM +0200, Reimar D?ffinger wrote:
> On Fri, Oct 08, 2010 at 07:49:35AM +0200, Reimar D?ffinger wrote:
> > On Fri, Oct 08, 2010 at 04:03:36AM +0200, Michael Niedermayer wrote:
> > > On Thu, Oct 07, 2010 at 09:35:37PM +0200, Reimar D?ffinger wrote:
> > > > Hello,
> > > > while it will leave the last line(s) uninitialized (do they even exist/will
> > > > they have been allocated?), below patch fixes the out-of-bounds reads.
> > > > Should fix the crash part of issue 2279.
> > > 
> > > they must have been allocated or there is a bug in the allocation
> > > and they should be filled
> > 
> > I'm not sure I understand really, but I think attached patch will
> > fill them (somewhat) correctly.
> > Note that the same issue (the uninitialized, not the crash part)
> > still exists for odd width AFAICT.
> > However for the given sample it at least solves all valgrind issues.
> 
> I think I can't think straight today.

carbon monoxid intoxication from the flames. get some fresh air ;)


> Below patch is much simpler, however it does not initialize any additional
> luma line the destination might have - this could be solved with a memcpy,
> but then we should document somewhere that a odd-height 4:2:0 actually
> must have that "additional" luma line.
> Index: libswscale/rgb2rgb_template.c

ok if tested

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Rewriting code that is poorly written but fully understood is good.
Rewriting code that one doesnt understand is a sign that one is less smart
then the original author, trying to rewrite it will not make it better.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20101008/2d1e366e/attachment.pgp>