[FFmpeg-devel] [PATCH 1/2] avcodec/x86/mpegenc: check IDCT permutation type is a valid value

[Michael Niedermayer]

On Fri, Jun 16, 2017 at 03:53:27PM +0200, James Darnley wrote:
> ---
>  libavcodec/x86/mpegvideoenc_template.c | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/libavcodec/x86/mpegvideoenc_template.c b/libavcodec/x86/mpegvideoenc_template.c
> index b2512744ca..3ce72e1367 100644
> --- a/libavcodec/x86/mpegvideoenc_template.c
> +++ b/libavcodec/x86/mpegvideoenc_template.c
> @@ -322,7 +322,7 @@ static int RENAME(dct_quantize)(MpegEncContext *s,
>          block[0x3E] = temp_block[0x3D]; block[0x33] = temp_block[0x36];
>          block[0x2F] = temp_block[0x2F]; block[0x37] = temp_block[0x37];
>          block[0x3B] = temp_block[0x3E]; block[0x3F] = temp_block[0x3F];
> -    }else{
> +    } else if (s->idsp.perm_type == FF_IDCT_PERM_NONE) {
>          if(last_non_zero_p1 <= 1) goto end;
>          block[0x01] = temp_block[0x01];
>          block[0x08] = temp_block[0x08]; block[0x10] = temp_block[0x10];
> @@ -366,6 +366,12 @@ static int RENAME(dct_quantize)(MpegEncContext *s,
>          block[0x3D] = temp_block[0x3D]; block[0x36] = temp_block[0x36];
>          block[0x2F] = temp_block[0x2F]; block[0x37] = temp_block[0x37];
>          block[0x3E] = temp_block[0x3E]; block[0x3F] = temp_block[0x3F];
> +    } else {
> +        av_log(s, AV_LOG_DEBUG, "s->idsp.perm_type: %d\n",
> +                (int)s->idsp.perm_type);
> +        av_assert0(s->idsp.perm_type == FF_IDCT_PERM_NONE ||
> +                s->idsp.perm_type == FF_IDCT_PERM_LIBMPEG2 ||
> +                s->idsp.perm_type == FF_IDCT_PERM_SIMPLE);
>      }

you could add the assert into the existing else case
that would be slightly simpler and might avoid a conditional branch

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

No snowflake in an avalanche ever feels responsible. -- Voltaire
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20170616/eb987aef/attachment.sig>