[FFmpeg-devel] [PATCH] lavu: check for overflow in av_clip_intp2_c

[Michael Niedermayer]

On Wed, Jan 13, 2016 at 12:52:21AM +0100, Andreas Cadhalpun wrote:
> ---
>  libavutil/common.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libavutil/common.h b/libavutil/common.h
> index f3276a2..5ae2847 100644
> --- a/libavutil/common.h
> +++ b/libavutil/common.h
> @@ -211,7 +211,7 @@ static av_always_inline av_const int32_t av_clipl_int32_c(int64_t a)
>   */
>  static av_always_inline av_const int av_clip_intp2_c(int a, int p)
>  {
> -    if ((a + (1 << p)) & ~((2 << p) - 1))
> +    if (a >= INT_MAX - (1 << p) || ((a + (1 << p)) & ~((2 << p) - 1)))

does this differ in any case from using unsigned for the addition?
if not then using unsigned avoids the additional operations

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety -- Benjamin Franklin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20160113/79b27e95/attachment.sig>