[FFmpeg-devel] [PATCH 3/3] avcodec/vvc/vvc_ps: check aps_adaptation_parameter_set_id

[James Almer]

On 1/26/2024 6:46 PM, Michael Niedermayer wrote:
> "When aps_params_type is equal to ALF_APS or SCALING_APS, the value of aps_adaptation_parameter_set_id shall be
> in the range of 0 to 7, inclusive.
> When aps_params_type is equal to LMCS_APS, the value of aps_adaptation_parameter_set_id shall be in the range of 0
> to 3, inclusive."

This should be checked in CBS instead, and not one of its users.
Something like:

> diff --git a/libavcodec/cbs_h266_syntax_template.c b/libavcodec/cbs_h266_syntax_template.c
> index 549d021211..9fd8399f0e 100644
> --- a/libavcodec/cbs_h266_syntax_template.c
> +++ b/libavcodec/cbs_h266_syntax_template.c
> @@ -2455,6 +2455,7 @@ static int FUNC(scaling_list_data)(CodedBitstreamContext *ctx, RWContext *rw,
>  static int FUNC(aps)(CodedBitstreamContext *ctx, RWContext *rw,
>                       H266RawAPS *current, int prefix)
>  {
> +    int aps_id_max = MAX_UINT_BITS(5);
>      int err;
> 
>      if (prefix)
> @@ -2467,7 +2468,12 @@ static int FUNC(aps)(CodedBitstreamContext *ctx, RWContext *rw,
>                                         : VVC_SUFFIX_APS_NUT));
> 
>      ub(3, aps_params_type);
> -    ub(5, aps_adaptation_parameter_set_id);
> +    if (current->aps_params_type == VVC_ASP_TYPE_ALF ||
> +        current->aps_params_type == VVC_ASP_TYPE_SCALING)
> +        aps_id_max = 7;
> +    else if (current->aps_params_type == VVC_ASP_TYPE_LMCS)
> +        aps_id_max = 3;
> +    u(5, aps_adaptation_parameter_set_id, 0, aps_id_max);
>      flag(aps_chroma_present_flag);
>      if (current->aps_params_type == VVC_ASP_TYPE_ALF)
>          CHECK(FUNC(alf_data)(ctx, rw, current));