[FFmpeg-devel] [PATCH 3/3] tiff: fix overflows when calling av_readuce

[Michael Niedermayer]

On Tue, Dec 13, 2016 at 12:50:19AM +0100, Andreas Cadhalpun wrote:
> The arguments of av_reduce are signed, so the cast to uint64_t is misleading.
> 
> Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun at googlemail.com>
> ---
>  libavcodec/tiff.c | 11 +++++++++--
>  1 file changed, 9 insertions(+), 2 deletions(-)
> 
> diff --git a/libavcodec/tiff.c b/libavcodec/tiff.c
> index 4721e94..12ef419 100644
> --- a/libavcodec/tiff.c
> +++ b/libavcodec/tiff.c
> @@ -772,9 +772,16 @@ static void set_sar(TiffContext *s, unsigned tag, unsigned num, unsigned den)
>      int offset = tag == TIFF_YRES ? 2 : 0;
>      s->res[offset++] = num;
>      s->res[offset]   = den;
> -    if (s->res[0] && s->res[1] && s->res[2] && s->res[3])
> +    if (s->res[0] && s->res[1] && s->res[2] && s->res[3]) {
> +        uint64_t num = s->res[2] * (uint64_t)s->res[1];
> +        uint64_t den = s->res[0] * (uint64_t)s->res[3];
> +        if (num > INT64_MAX || den > INT64_MAX) {
> +            num = num >> 1;
> +            den = den >> 1;
> +        }

this can make one of them 0, in fact i think even if they arent 0
the sample_aspect_ratio can be  after reduce
should they be checked after all that instead of before ?


[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Modern terrorism, a quick summary: Need oil, start war with country that
has oil, kill hundread thousand in war. Let country fall into chaos,
be surprised about raise of fundamantalists. Drop more bombs, kill more
people, be surprised about them taking revenge and drop even more bombs
and strip your own citizens of their rights and freedoms. to be continued
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20161213/b4f19e56/attachment.sig>