[FFmpeg-devel] [PATCH] libopenjpegenc: fix out-of-bounds reads when filling the edges

[Andreas Cadhalpun]

On 14.10.2016 00:49, Michael Niedermayer wrote:
> On Fri, Oct 14, 2016 at 12:23:02AM +0200, Andreas Cadhalpun wrote:
>> The avctx->width/avctx->height is not zero, but libopenjpeg_copy_unpacked8
>> does:
> 
>>         width  = avctx->width / image->comps[compno].dx;
>>         height = avctx->height / image->comps[compno].dy;
> 
> this looks wrong to me
> the code in mj2_create_image() looks better:
>         cmptparm[i].dx = sub_dx[i];
>         cmptparm[i].dy = sub_dy[i];
>         cmptparm[i].w = (avctx->width + sub_dx[i] - 1) / sub_dx[i];
>         cmptparm[i].h = (avctx->height + sub_dy[i] - 1) / sub_dy[i];

Indeed this looks better, so I updated the patch (attached) to change the
calculation of width/height.

Best regards,
Andreas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-libopenjpegenc-fix-out-of-bounds-reads-when-filling-.patch
Type: text/x-diff
Size: 4536 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20161014/f369a19e/attachment.patch>