[FFmpeg-devel] [PATCH] libopenjpegenc: fix out-of-bounds reads when filling the edges

[Michael Niedermayer]

On Fri, Oct 14, 2016 at 02:00:49AM +0200, Andreas Cadhalpun wrote:
> On 14.10.2016 00:49, Michael Niedermayer wrote:
> > On Fri, Oct 14, 2016 at 12:23:02AM +0200, Andreas Cadhalpun wrote:
> >> The avctx->width/avctx->height is not zero, but libopenjpeg_copy_unpacked8
> >> does:
> > 
> >>         width  = avctx->width / image->comps[compno].dx;
> >>         height = avctx->height / image->comps[compno].dy;
> > 
> > this looks wrong to me
> > the code in mj2_create_image() looks better:
> >         cmptparm[i].dx = sub_dx[i];
> >         cmptparm[i].dy = sub_dy[i];
> >         cmptparm[i].w = (avctx->width + sub_dx[i] - 1) / sub_dx[i];
> >         cmptparm[i].h = (avctx->height + sub_dy[i] - 1) / sub_dy[i];
> 
> Indeed this looks better, so I updated the patch (attached) to change the
> calculation of width/height.
> 
> Best regards,
> Andreas

>  libopenjpegenc.c |   18 +++++++++---------
>  1 file changed, 9 insertions(+), 9 deletions(-)
> 17061aee3e88729993c9581f688cbfda01fccaac  0001-libopenjpegenc-fix-out-of-bounds-reads-when-filling-.patch
> From 1461064c1eaabb71661f9ff68b94f35a1b98e3b5 Mon Sep 17 00:00:00 2001
> From: Andreas Cadhalpun <Andreas.Cadhalpun at googlemail.com>
> Date: Thu, 13 Oct 2016 22:14:46 +0200
> Subject: [PATCH] libopenjpegenc: fix out-of-bounds reads when filling the
>  edges
> 
> The calculation of width/height should round up, not round down to
> prevent setting width or height to 0.
> 
> Also image->comps[compno].w is unsigned (at least in openjpeg2), so the
> calculation could silently wrap around without the explicit cast to int.

LGTM, iam not libopenjpegenc maintainer though

also should be backported

thx

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

It is what and why we do it that matters, not just one of them.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20161014/a57052dc/attachment.sig>