[FFmpeg-devel] [PATCH 1/5] avcodec/apedec: Fix 32bit int overflow in do_apply_filter()

[Michael Niedermayer]

On Tue, Sep 03, 2019 at 06:58:03PM +0200, Tomas Härdin wrote:
> tis 2019-09-03 klockan 13:25 +0200 skrev Michael Niedermayer:
> > On Tue, Sep 03, 2019 at 12:16:47PM +0200, Tomas Härdin wrote:
> > > tis 2019-09-03 klockan 02:14 +0200 skrev Michael Niedermayer:
> > > > Fixes: signed integer overflow: 2147480546 + 4096 cannot be represented in type 'int'
> > > > Fixes: 16280/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-5123442566758400
> > > > 
> > > > Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> > > > Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
> > > > ---
> > > >  libavcodec/apedec.c | 2 +-
> > > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > > 
> > > > diff --git a/libavcodec/apedec.c b/libavcodec/apedec.c
> > > > index 490b11b94e..ed22f0f019 100644
> > > > --- a/libavcodec/apedec.c
> > > > +++ b/libavcodec/apedec.c
> > > > @@ -1266,7 +1266,7 @@ static void do_apply_filter(APEContext *ctx, int version, APEFilter *f,
> > > >                                                       f->delay - order,
> > > >                                                       f->adaptcoeffs - order,
> > > >                                                       order, APESIGN(*data));
> > > > -        res = (res + (1 << (fracbits - 1))) >> fracbits;
> > > > +        res = (int)(res + (1U << (fracbits - 1))) >> fracbits;
> > > 
> > > Does this mean the old code was decoding incorrectly, or is this just
> > > an UB fix? (fixing UBs is good of course)
> > 
> > it is just a UB fix.
> > If iam not misreading the reference source then it has the same UB in it
> > so there is no correct way in case this ever triggers on a real file.
> > 
> > The specific patch here was done so that the result should hopefully
> > match what a common machine with 32bit int (most platforms) would
> > likely produce.
> 
> Right, and the cast back to int is required for proper sign extension
> in the >> fracbits. Looks OK to me then. Kind of a weird codec feature
> IMO

ok, will apply

thanks

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

If you think the mosad wants you dead since a long time then you are either
wrong or dead since a long time.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20190905/490fdbc5/attachment.sig>