[FFmpeg-devel] [PATCH] avcodec/ttaenc: Fix undefined shift
Paul B Mahol
onemda at gmail.com
Mon Sep 16 11:16:36 EEST 2019
LGTM
On 9/15/19, Andreas Rheinhardt <andreas.rheinhardt at gmail.com> wrote:
> ttaenc contained (1 << unary) - 1 as an argument for a function
> expecting an unsigned int. unary can be as big as 31 in this case.
> The type of the shift and the whole expression is int, because 1 fits
> into an integer, so that the behaviour is undefined if unary == 31
> as the result of the shift can't be represented in an int §. Subtraction
> by 1 (which makes the result of the whole expression representable in
> an int) doesn't change that this is undefined (it usually leads to
> signed integer overflow which is undefined, too).
>
> The solution is simple: Make 1 unsigned to change the type of the
> whole expression to unsigned int (as the function expects anyway).
>
> Fixes ticket #8153.
>
> §: This of course presupposes the common int range of -2^31..2^31-1
>
> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt at gmail.com>
> ---
> libavcodec/ttaenc.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/libavcodec/ttaenc.c b/libavcodec/ttaenc.c
> index 3cc54d78c5..08a0d0483a 100644
> --- a/libavcodec/ttaenc.c
> +++ b/libavcodec/ttaenc.c
> @@ -164,7 +164,7 @@ pkt_alloc:
> put_bits(&pb, 31, 0x7FFFFFFF);
> unary -= 31;
> } else {
> - put_bits(&pb, unary, (1 << unary) - 1);
> + put_bits(&pb, unary, (1U << unary) - 1);
> unary = 0;
> }
> } while (unary);
> --
> 2.21.0
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
More information about the ffmpeg-devel
mailing list