[FFmpeg-devel] [PATCH] lavc/qsvenc: replace assert0 with an error return when pict_type is uninitialized

[Carl Eugen Hoyos]

2018-12-05 6:37 GMT+01:00, Li, Zhong <zhong.li at intel.com>:
>> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On Behalf
>> Of Linjie Fu
>> Sent: Monday, December 3, 2018 4:34 PM
>> To: ffmpeg-devel at ffmpeg.org
>> Cc: Fu, Linjie <linjie.fu at intel.com>
>> Subject: [FFmpeg-devel] [PATCH] lavc/qsvenc: replace assert0 with an
>> error
>> return when pict_type is uninitialized
>>
>> pict_type may be uninitialized, and assert on a value coming from an
>> external library is not proper.
>>
>> Return invalid data error in function ff_qsv_encode to avoid using
>> uninitialized value.
>>
>> Signed-off-by: Linjie Fu <linjie.fu at intel.com>
>> ---
>>  libavcodec/qsvenc.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c index
>> 7f4592f878..891253be76 100644
>> --- a/libavcodec/qsvenc.c
>> +++ b/libavcodec/qsvenc.c
>> @@ -1338,7 +1338,7 @@ int ff_qsv_encode(AVCodecContext *avctx,
>> QSVEncContext *q,
>>          else if (bs->FrameType & MFX_FRAMETYPE_B || bs->FrameType
>> & MFX_FRAMETYPE_xB)
>>              pict_type = AV_PICTURE_TYPE_B;
>>          else
>> -            av_assert0(!"Uninitialized pict_type!");
>> +            return AVERROR_INVALIDDATA;
>
> Thus will be still a broken issue of ticket # 7593.

But an important - and long discussed - bug would be replaced by a
graceful exit.

Carl Eugen