[FFmpeg-devel] [PATCH 2/2] avcodec/pixlet: Fixes: undefined shift in av_mod_uintp2()

[Michael Niedermayer]

On Sun, Aug 20, 2017 at 04:06:46PM +0200, Paul B Mahol wrote:
> On 8/19/17, Michael Niedermayer <michael at niedermayer.cc> wrote:
> > On Sat, Aug 19, 2017 at 04:00:18PM +0200, Paul B Mahol wrote:
> >> On 8/19/17, Michael Niedermayer <michael at niedermayer.cc> wrote:
> >> > On Fri, Aug 18, 2017 at 06:21:56PM +0200, Paul B Mahol wrote:
> >> >> On 8/18/17, Michael Niedermayer <michael at niedermayer.cc> wrote:
> >> >> > Fixes: runtime error: shift exponent 4294967289 is too large for
> >> >> > 32-bit
> >> >> > type
> >> >> > 'int'
> >> >> > Fixes: 3030/clusterfuzz-testcase-minimized-4649809254285312
> >> >> >
> >> >> > Found-by: continuous fuzzing process
> >> >> > https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> >> >> > Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
> >> >> > ---
> >> >> >  libavcodec/pixlet.c | 2 +-
> >> >> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >> >> >
> >> >> > diff --git a/libavcodec/pixlet.c b/libavcodec/pixlet.c
> >> >> > index 088226bdda..a9cfe085c9 100644
> >> >> > --- a/libavcodec/pixlet.c
> >> >> > +++ b/libavcodec/pixlet.c
> >> >> > @@ -262,7 +262,7 @@ static int read_high_coeffs(AVCodecContext
> >> >> > *avctx,
> >> >> > uint8_t *src, int16_t *dst, i
> >> >> >
> >> >> >          flag = 0;
> >> >> >
> >> >> > -        if (state * 4ULL > 0xFF || i >= size)
> >> >> > +        if ((uint64_t)state > 0xFF / 4 || i >= size)
> >> >>
> >> >> This is not exact same behaviour.
> >> >
> >> > no, it differs for a small number of cases that have one of the 2
> >> > most significant bits (of the 64 bits) set in state.
> >> > I think this was a bug i introduced in a previous commit.
> >> >
> >> > If above change is not the correct thing to do, please elaborate.
> >> > state is passed into ff_clz() later which has no deterministic
> >> > behavior when these MSBs are set as ff_clz() uses unsigned and state
> >> > is int64_t and unsigned can be 32 or 64bit.
> >>
> >> Only 32 bits matters for ff_clz() anyway.
> >
> > your reply is a bit terse.
> > Does that mean you have no objections to the patch ?
> >
> > about ff_clz(), with 64bit unsigned a ff_clz(1) should return 63, the
> > code looks like it expects 31
> 
> yes.

will apply the 2 patches

thanks

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

I have often repented speaking, but never of holding my tongue.
-- Xenocrates
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20170820/c9675101/attachment.sig>