[FFmpeg-devel] [PATCH] avcodec/cabac: do not use signed values for ff_h264_cabac_tables

[Nuo Mi]

On Sun, Mar 21, 2021 at 12:10 PM Nuo Mi <nuomi2021 at gmail.com> wrote:

> The type of ff_h264_cabac_tables in unit8, it's not signed.
> Also,H.264 Table 9-44 and H.265 Table 9-52 never used a signed value
> ---
>  libavcodec/cabac.c | 20 ++++++++++----------
>  1 file changed, 10 insertions(+), 10 deletions(-)
>
> diff --git a/libavcodec/cabac.c b/libavcodec/cabac.c
> index 6649feebeb..fe6bac3d47 100644
> --- a/libavcodec/cabac.c
> +++ b/libavcodec/cabac.c
> @@ -54,7 +54,7 @@ DECLARE_ASM_ALIGNED(1, const uint8_t,
> ff_h264_cabac_tables)[512 + 4*2*64 + 4*64
>      0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>      0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
>      // LPS range
> -    -128,    -128,    -128,    -128,    -128,    -128,    123,     123,
> +    128,     128,     128,     128,     128,     128,     123,     123,
>      116,     116,     111,     111,     105,     105,     100,     100,
>      95,      95,      90,      90,      85,      85,      81,      81,
>      77,      77,      73,      73,      69,      69,      66,      66,
> @@ -70,8 +70,8 @@ DECLARE_ASM_ALIGNED(1, const uint8_t,
> ff_h264_cabac_tables)[512 + 4*2*64 + 4*64
>      10,      10,      9,       9,       9,       9,       8,       8,
>      8,       8,       7,       7,       7,       7,       7,       7,
>      6,       6,       6,       6,       6,       6,       2,       2,
> -    -80,     -80,     -89,     -89,     -98,     -98,     -106,    -106,
> -    -114,    -114,    -121,    -121,    -128,    -128,    122,     122,
> +    176,     176,     167,     167,     158,     158,     150,     150,
> +    142,     142,     135,     135,     128,     128,     122,     122,
>      116,     116,     110,     110,     104,     104,     99,      99,
>      94,      94,      89,      89,      85,      85,      80,      80,
>      76,      76,      72,      72,      69,      69,      65,      65,
> @@ -86,9 +86,9 @@ DECLARE_ASM_ALIGNED(1, const uint8_t,
> ff_h264_cabac_tables)[512 + 4*2*64 + 4*64
>      12,      12,      11,      11,      11,      11,      10,      10,
>      9,       9,       9,       9,       9,       9,       8,       8,
>      8,       8,       7,       7,       7,       7,       2,       2,
> -    -48,     -48,     -59,     -59,     -69,     -69,     -78,     -78,
> -    -87,     -87,     -96,     -96,     -104,    -104,    -112,    -112,
> -    -119,    -119,    -126,    -126,    123,     123,     117,     117,
> +    208,     208,     197,     197,     187,     187,     178,     178,
> +    169,     169,     160,     160,     152,     152,     144,     144,
> +    137,     137,     130,     130,     123,     123,     117,     117,
>      111,     111,     105,     105,     100,     100,     95,      95,
>      90,      90,      86,      86,      81,      81,      77,      77,
>      73,      73,      69,      69,      66,      66,      63,      63,
> @@ -102,10 +102,10 @@ DECLARE_ASM_ALIGNED(1, const uint8_t,
> ff_h264_cabac_tables)[512 + 4*2*64 + 4*64
>      14,      14,      13,      13,      12,      12,      12,      12,
>      11,      11,      11,      11,      10,      10,      10,      10,
>      9,       9,       9,       9,       8,       8,       2,       2,
> -    -16,     -16,     -29,     -29,     -40,     -40,     -51,     -51,
> -    -61,     -61,     -71,     -71,     -81,     -81,     -90,     -90,
> -    -98,     -98,     -106,    -106,    -114,    -114,    -121,    -121,
> -    -128,    -128,    122,     122,     116,     116,     110,     110,
> +    240,     240,     227,     227,     216,     216,     205,     205,
> +    195,     195,     185,     185,     175,     175,     166,     166,
> +    158,     158,     150,     150,     142,     142,     135,     135,
> +    128,     128,     122,     122,     116,     116,     110,     110,
>      104,     104,     99,      99,      94,      94,      89,      89,
>      85,      85,      80,      80,      76,      76,      72,      72,
>      69,      69,      65,      65,      62,      62,      59,      59,
> --
> 2.25.1
>
This passed following fate test.

make fate-h264
make fate-hevc