[Ffmpeg-devel] [PATCH] ported SGI decoder to the new API

[Michael Niedermayer]

Hi

On Tue, Mar 27, 2007 at 10:10:27AM +0800, Xiaohui Sun wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Michael Niedermayer wrote:
> > Hi
> > 
> > On Mon, Mar 26, 2007 at 10:28:30AM +0800, Xiaohui Sun wrote:
> > [...]
> >>> [...]
> >>>> +
> >>>> + for (z = 0; z < zsize; z++) {
> >>>> + for (y = 0; y < ysize; y++) {
> >>>> + dest_row = ptr + (ysize - 1 - y) * (xsize * zsize);
> >>> whatever this is supposed to do the multiplication can overflow
> >>> which considering this is a destination pointer to write is not good
> >> then where should the decoded buffer be written to.
> > 
> > whereever you like, just not in unallocated memory
> 
> I have allocated buffer in line 234 : avctx->get_buffer(avctx, p) I am
> not is this the right place to do the allocation?

the problem is that the multiplication can overflow like
ysize=65535
xsize=65535
zsize=3
y=0
(ysize - 1 - y) * (xsize * zsize)= 12884312070
(int)12884312070 = -589818

which when added with ptr points prior to the array

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The educated differ from the uneducated as much as the living from the
dead. -- Aristotle 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20070327/c88a4f55/attachment.pgp>