[FFmpeg-devel] [PATCH 1/5] avformat/aviobuf: Don't check for overflow after it happened

[James Almer]

On 5/25/2020 11:07 AM, Andreas Rheinhardt wrote:
> If adding two ints overflows, it doesn't matter whether the result will
> be stored in an unsigned or not; and checking afterwards does not make it
> retroactively defined.
> 
> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt at gmail.com>
> ---
>  libavformat/aviobuf.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libavformat/aviobuf.c b/libavformat/aviobuf.c
> index eb0387bdf7..33c2d6f037 100644
> --- a/libavformat/aviobuf.c
> +++ b/libavformat/aviobuf.c
> @@ -1275,7 +1275,7 @@ static int dyn_buf_write(void *opaque, uint8_t *buf, int buf_size)
>      unsigned new_size, new_allocated_size;
>  
>      /* reallocate buffer if needed */
> -    new_size = d->pos + buf_size;
> +    new_size = (unsigned)d->pos + buf_size;
>      new_allocated_size = d->allocated_size;
>      if (new_size < d->pos || new_size > INT_MAX/2)
>          return -1;

You could instead do

if (d->pos > INT_MAX / 2 - buf_size)
    return -1;
new_size = d->pos + buf_size;

Should also work fine with the changes in 3/5 adapted to it.