[FFmpeg-devel] [PATCH] lavc/vvc: Clamp shift RHS

[James Almer]

On 1/26/2024 6:13 AM, post at frankplowman.com wrote:
> From: Frank Plowman <post at frankplowman.com>
> 
> Resolves the following undefined behavior sanitiser error:
> runtime error: shift exponent 32 is too large for 32-bit type 'int'
> 
> Signed-off-by: Frank Plowman <post at frankplowman.com>
> ---
>   libavcodec/vvc/vvc_intra_template.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libavcodec/vvc/vvc_intra_template.c b/libavcodec/vvc/vvc_intra_template.c
> index 9fb47549d5..a078885266 100644
> --- a/libavcodec/vvc/vvc_intra_template.c
> +++ b/libavcodec/vvc/vvc_intra_template.c
> @@ -969,7 +969,7 @@ static void FUNC(pred_angular_h)(uint8_t *_src, const uint8_t *_top, const uint8
>           int pos = (1 + ref_idx) * intra_pred_angle;
>           int wt;
>           if (need_pdpc)
> -            wt = (32 >> ((y * 2) >> nscale));
> +            wt = (32 >> FFMIN(31, (y * 2) >> nscale));
>   
>           for (int x = 0; x < w; x++) {
>               const int idx  = (pos >> 5) + ref_idx;

Applied, thanks.