[Ffmpeg-devel] [PATCH] ported SGI decoder to the new API

[Xiaohui Sun]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Michael Niedermayer wrote:
> Hi
> 

> 
>>> [...]
>>>> +            start_offset = bytestream_get_be32(&start_table);
>>>> +            if(start_offset > end_buf - in_buf) {
>>>> +                return AVERROR_INVALIDDATA;
>>>> +            }
>>> excelent, finally the check looks good
>>>
>>>
>>>> +            if (expand_rle_row(in_buf + start_offset, end_buf, 
>>> dest_row,
>>>> +                dest_row + s->linesize, z, s->depth) != s->width)
>>>> +                return AVERROR_INVALIDDATA;
>>> linesize can be negative
>>>
>> I modified, but not sure.
> 
> it should have been dest_row + width*pixelsize or dest_row + FFABS(s->linesize)

I found width*pixelsize is not equal to(actually less than)
FFABS(s->linesize). because I found in avcodec_default_get_buffer it
will add some extra EDGE_WIDTH to it, is that right?


[...]

> [...]
>> +    out_ptr = p->data[0];
>> +
>> +    end_ptr = out_ptr + p->linesize[0] * s->height;
>> +
>> +    if(p->linesize[0] < 0) {
>> +        s->linesize = - p->linesize[0];
>> +        FFSWAP(uint8_t*, end_ptr, out_ptr);
>> +    } else {
>> +        s->linesize = p->linesize[0];
>> +    }
> 
> this is wrong
> first, it simply wont work as the pointers are off by one line
> second, if the pointers where not off by one line it would store a
> vertically fliped image

so we simply make an abstract of linesize without the need of adjusting
corresponding pointers?

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGEvnb+BwsLu3sVWwRAjSOAJsF5aO6vUuAtVUuMlmqYBQ46bzNbACghH5p
ae5TSVz+uLWpdJy4I3pgxQ8=
=jzdX
-----END PGP SIGNATURE-----