[FFmpeg-devel] [PATCH] swscale: avoid overread in planar2x

[Michael Niedermayer]

On Sun, Sep 12, 2010 at 12:11:20PM +0200, Michael Niedermayer wrote:
> On Sun, Sep 12, 2010 at 12:11:58AM -0300, Ramiro Polla wrote:
> > Hi,
> > 
> > The MMX2/3dnow code in planar2x in rgb2rgb currently reads one byte
> > prior to the source buffer. This leads to a crash on Windows and Mac
> > OS X when the buffer is allocated on the beginning of a page. This
> > should also be a problem on Linux but I haven't seen many crashes
> > because of overreading...
> 
> the code looks buggy
> the first pixel should be [0] not [-1] that can be achived by
> moving the reading code of it to the end of the loop and constructing
> the contents of the first mm4 outside the loop

after a second look i see that the C code overrides the first pixel and thus
the output should not be affected, still my suggestion though seems nicer than
requireing all buffers to have a byte allocated before them

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

it is not once nor twice but times without number that the same ideas make
their appearance in the world. -- Aristotle
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20100912/4d24f0bb/attachment.pgp>