[FFmpeg-devel] [PATCH] lavfi/alphaextract: fix invalid buffer access in case of negative YUV linesize

[Stefano Sabatini]

On date Friday 2012-12-07 02:52:52 +0100, Clément Bœsch encoded:
> On Fri, Dec 07, 2012 at 12:06:13AM +0100, Stefano Sabatini wrote:
> > Fix crash.
> > ---
> >  libavfilter/vf_alphaextract.c |   18 +++++++++++++-----
> >  1 file changed, 13 insertions(+), 5 deletions(-)
> > 
> > diff --git a/libavfilter/vf_alphaextract.c b/libavfilter/vf_alphaextract.c
> > index 766cc8c..94da122 100644
> > --- a/libavfilter/vf_alphaextract.c
> > +++ b/libavfilter/vf_alphaextract.c
> > @@ -85,14 +85,22 @@ static int filter_frame(AVFilterLink *inlink, AVFilterBufferRef *cur_buf)
> >          }
> >      } else if (cur_buf->linesize[A] == out_buf->linesize[Y]) {
> >          const int linesize = cur_buf->linesize[A];
> > -        memcpy(out_buf->data[Y], cur_buf->data[A], linesize * inlink->h);
> > +        const int blocksize = abs(linesize)*(inlink->h);
> > +        if (linesize < 0)
> > +            memcpy(out_buf->data[Y]-blocksize+linesize,
> > +                   cur_buf->data[A]-blocksize+linesize*(inlink->h-1), blocksize);
> > +        else
> > +            memcpy(out_buf->data[Y], cur_buf->data[A], blocksize);
> >      } else {
> > -        const int linesize = FFMIN(out_buf->linesize[Y], cur_buf->linesize[A]);
> > +        const int linesize = abs(FFMIN(out_buf->linesize[Y], cur_buf->linesize[A]));
> > +        uint8_t *pout = out_buf->data[Y];
> > +        uint8_t *pin  = cur_buf->data[A];
> >          int y;
> > +
> >          for (y = 0; y < inlink->h; y++) {
> > -            memcpy(out_buf->data[Y] + y * out_buf->linesize[Y],
> > -                   cur_buf->data[A] + y * cur_buf->linesize[A],
> > -                   linesize);
> > +            memcpy(pout, pin, linesize);
> > +            pout += out_buf->linesize[Y];
> > +            pin  += cur_buf->linesize[A];
> 
> As said in the comment from the previous patch: won't it make sense to
> merge the two conditional blocks?
> 

> Also, I wonder if some other filters are not affected by this bug (it
> might make sense to rejection the negative linesize perm for them): how
> did you test?

Don't know, but sure I was sleeping when I reviewed this one.

Patch updated.
-- 
FFmpeg = Friendly and Faithless Mythic Political Ecletic Guru
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-lavfi-alphaextract-fix-assignment-of-invalid-value-t.patch
Type: text/x-diff
Size: 1227 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20121207/a28099ae/attachment.bin>