[FFmpeg-devel] [PATCH v2] avfilter/scale: fix CID 1457833

[Michael Niedermayer]

On Fri, Jan 31, 2020 at 12:46:13PM +0530, Gyan Doshi wrote:
> Remove expressions with constant results and
> improve overflow checks.
> ---
>  libavfilter/vf_scale.c | 6 ++----
>  1 file changed, 2 insertions(+), 4 deletions(-)
> 
> diff --git a/libavfilter/vf_scale.c b/libavfilter/vf_scale.c
> index 0348f19d33..b6c6414258 100644
> --- a/libavfilter/vf_scale.c
> +++ b/libavfilter/vf_scale.c
> @@ -497,10 +497,8 @@ static int config_props(AVFilterLink *outlink)
>                                 scale->force_original_aspect_ratio,
>                                 scale->force_divisible_by);
>  
> -    if (scale->w > INT_MAX ||
> -        scale->h > INT_MAX ||
> -        (scale->h * inlink->w) > INT_MAX ||
> -        (scale->w * inlink->h) > INT_MAX)
> +    if ((scale->h > INT_MAX / inlink->w) ||
> +        (scale->w > INT_MAX / inlink->h))
>          av_log(ctx, AV_LOG_ERROR, "Rescaled value for width or height is too big.\n");

taking a very quick look at the history it seemed the w / h values where
originally int64_t

more specifically, av_rescale() produces int64_t and simply storing that
in int might introduce a truncation of the results so there are maybe
checks missing elsewhere since the variables changed to int

Thanks

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Frequently ignored answer#1 FFmpeg bugs should be sent to our bugtracker. User
questions about the command line tools should be sent to the ffmpeg-user ML.
And questions about how to use libav* should be sent to the libav-user ML.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20200131/e5a770d4/attachment.sig>