[FFmpeg-devel] [PATCH 3/3] avcodec/hevc_ps: Avoid signed overflow before check on QP

[James Almer]

On 4/27/2023 3:38 PM, Michael Niedermayer wrote:
> Fixes: signed integer overflow: -2147483648 - 5 cannot be represented in type 'int'
> Fixes: 58066/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HEVC_fuzzer-5312995835379712
> 
> Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
> ---
>   libavcodec/hevc_ps.c | 6 +++---
>   1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/libavcodec/hevc_ps.c b/libavcodec/hevc_ps.c
> index 1533e2a817..6b8f432609 100644
> --- a/libavcodec/hevc_ps.c
> +++ b/libavcodec/hevc_ps.c
> @@ -1522,9 +1522,9 @@ static int pps_scc_extension(GetBitContext *gb, AVCodecContext *avctx,
>       pps->pps_curr_pic_ref_enabled_flag = get_bits1(gb);
>       if (pps->residual_adaptive_colour_transform_enabled_flag = get_bits1(gb)) {
>           pps->pps_slice_act_qp_offsets_present_flag = get_bits1(gb);
> -        pps->pps_act_y_qp_offset  = get_se_golomb_long(gb) - 5;
> -        pps->pps_act_cb_qp_offset = get_se_golomb_long(gb) - 5;
> -        pps->pps_act_cr_qp_offset = get_se_golomb_long(gb) - 3;
> +        pps->pps_act_y_qp_offset  = get_se_golomb_long(gb) - 5U;
> +        pps->pps_act_cb_qp_offset = get_se_golomb_long(gb) - 5U;
> +        pps->pps_act_cr_qp_offset = get_se_golomb_long(gb) - 3U;

Spec compliant values for all of these are in the -7..17 and -9..15 
range, so just use get_se_golomb() instead, which i assume is for small 
values, much like get_ue_golomb().

>   
>   #define CHECK_QP_OFFSET(name) (pps->pps_act_ ## name ## _qp_offset <= -12 || \
>                                  pps->pps_act_ ## name ## _qp_offset >= 12)