[FFmpeg-devel] [PATCH] libopenjpegenc: fix out-of-bounds reads when filling the edges

[Michael Niedermayer]

On Fri, Oct 14, 2016 at 12:23:02AM +0200, Andreas Cadhalpun wrote:
> On 14.10.2016 00:00, Hendrik Leppkes wrote:
> > On Thu, Oct 13, 2016 at 10:25 PM, Andreas Cadhalpun
> > <andreas.cadhalpun at googlemail.com> wrote:
> >> If x is 0, 'x - 1' is in the previous line, or worse outside the buffer
> >> for the first line.
> >>
> >> If y is 0, 'x - image->comps[compno].w' is outside the buffer.
> >>
> > 
> > I'm slightly puzzled, as you say, these are for edge handling, edges
> > in this case are from the image width to buffer width, and image
> > height to buffer height, respectively
> > So for x or y to be zero, we would need an image thats zero width, or
> > zero height, so the edge starts at zero?
> > 
> > How does that happen, and wouldn't it be much simpler to catch that
> > case earlier in the chain and simply error out? A image with either
> > zero width or zero height surely is not something you can encode
> > either way.
> 
> The avctx->width/avctx->height is not zero, but libopenjpeg_copy_unpacked8
> does:

>         width  = avctx->width / image->comps[compno].dx;
>         height = avctx->height / image->comps[compno].dy;

this looks wrong to me
the code in mj2_create_image() looks better:
        cmptparm[i].dx = sub_dx[i];
        cmptparm[i].dy = sub_dy[i];
        cmptparm[i].w = (avctx->width + sub_dx[i] - 1) / sub_dx[i];
        cmptparm[i].h = (avctx->height + sub_dy[i] - 1) / sub_dy[i];

i assume here that the buffers are large enough and  that the encoder
encodes the full rounded up size
if it does not then it doesnt support odd sizes with subsampling


> 
> So if e.g. avctx->height is 1 and image->comps[compno].dy is 2, height
> becomes 0. I'm not sure if that's invalid.

chroma planes being 0 sized is invalid for a normal image

you can potentially get this with tiling that some tiles have no
chroma samples but i think the code doesnt do anything odd like that
of course i could also be missing somethig 

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The real ebay dictionary, page 1
"Used only once"    - "Some unspecified defect prevented a second use"
"In good condition" - "Can be repaird by experienced expert"
"As is" - "You wouldnt want it even if you were payed for it, if you knew ..."
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20161014/82f03aec/attachment.sig>