[FFmpeg-devel] [PATCH 12/19] avcodec/mpegvideo_enc: Don't segfault on unorthodox mpeg_quant

[Andreas Rheinhardt]

Michael Niedermayer:
> On Tue, Apr 06, 2021 at 11:55:09PM +0200, Andreas Rheinhardt wrote:
>> The (deprecated) field AVCodecContext.mpeg_quant has no range
>> restriction; MpegEncContext.mpeg_quant is restricted to 0..1.
>> If the former is set, the latter is overwritten with it without
>> checking the range. This can trigger an av_assert2() with the MPEG-4
>> encoder when writing said field.
>>
>> Fix this by just setting MpegEncContext.mpeg_quant to 1 if
>> AVCodecContext.mpeg_quant is set.
>>
>> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt at outlook.com>
>> ---
>> This can not be triggered if one only sets options via the dictionary in
>> avcodec_open2(); one needs to set it directly in the AVCodecContext or
>> use the private class of AVCodecContext.
> 
> I tend toward checking mpeg_quant to be valid in init somewhere but this
> here is fine too or in addition
> 
Do you want to error out if AVCodecContext.mpeg_quant is outside of
0..1? Or do you want another check for MpegEncContext.mpeg_quant? The
option for the latter already uses the proper range, so I don't think we
need a check for that.

- Andreas