[FFmpeg-devel] [PATCH v4 1/9] cbs_av1: fix incorrect data type

[Wang, Fei W]

On Fri, 2021-07-09 at 11:32 +0800, Fei Wang wrote:
> Since order_hint_bits_minus_1 range is 0~7, cur_frame_hint can be
> most 128. And similar return value for cbs_av1_get_relative_dist.
> So if plus them and use int8_t for the result may lose its precision.
> 
> Signed-off-by: Fei Wang <fei.w.wang at intel.com>
> ---
>  libavcodec/cbs_av1_syntax_template.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libavcodec/cbs_av1_syntax_template.c
> b/libavcodec/cbs_av1_syntax_template.c
> index 6fe6e9a4f3..d98d3d42de 100644
> --- a/libavcodec/cbs_av1_syntax_template.c
> +++ b/libavcodec/cbs_av1_syntax_template.c
> @@ -355,7 +355,7 @@ static int
> FUNC(set_frame_refs)(CodedBitstreamContext *ctx, RWContext *rw,
>          AV1_REF_FRAME_ALTREF2, AV1_REF_FRAME_ALTREF
>      };
>      int8_t ref_frame_idx[AV1_REFS_PER_FRAME],
> used_frame[AV1_NUM_REF_FRAMES];
> -    int8_t shifted_order_hints[AV1_NUM_REF_FRAMES];
> +    int16_t shifted_order_hints[AV1_NUM_REF_FRAMES];
>      int cur_frame_hint, latest_order_hint, earliest_order_hint, ref;
>      int i, j;
>  

Hi James,

This version fixed all comments from community, could you help to
review?

Thanks
Fei