[FFmpeg-devel] [PATCH 1/2] fftools/ffmpeg_filter: fix null pointer dereference

[Zhao Zhili]

> On Dec 18, 2023, at 20:04, Zhao Zhili <quinkblack at foxmail.com> wrote:
> 
> 
> 
>> On Dec 18, 2023, at 19:19, Anton Khirnov <anton at khirnov.net> wrote:
>> 
>> Quoting Zhao Zhili (2023-12-17 12:01:08)
>>> From: Zhao Zhili <zhilizhao at tencent.com>
>>> 
>>> A dummy frame is created with format NONE passed to enc_open(),
>>> which doesn't prepare for it. The null pointer dereference happened
>>> at av_pix_fmt_desc_get(enc_ctx->pix_fmt)->comp[0].depth.
>>> 
>>> frame #0: 0x0000005555bc34a4 ffmpeg_g`enc_open(opaque=0xb400007efe2db690, frame=0xb400007efe2d9f70) at ffmpeg_enc.c:235:44
>>> frame #1: 0x0000005555bef250 ffmpeg_g`enc_open(sch=0xb400007dde2d4090, enc=0xb400007e4e2daad0, frame=0xb400007efe2d9f70) at ffmpeg_sched.c:1462:11
>>> frame #2: 0x0000005555bee094 ffmpeg_g`send_to_enc(sch=0xb400007dde2d4090, enc=0xb400007e4e2daad0, frame=0xb400007efe2d9f70) at ffmpeg_sched.c:1571:19
>>> frame #3: 0x0000005555bee01c ffmpeg_g`sch_filter_send(sch=0xb400007dde2d4090, fg_idx=0, out_idx=0, frame=0xb400007efe2d9f70) at ffmpeg_sched.c:2154:12
>>> frame #4: 0x0000005555bcf124 ffmpeg_g`close_output(ofp=0xb400007e4e2d85b0, fgt=0x0000007d1790eb08) at ffmpeg_filter.c:2225:15
>>> frame #5: 0x0000005555bcb000 ffmpeg_g`fg_output_frame(ofp=0xb400007e4e2d85b0, fgt=0x0000007d1790eb08, frame=0x0000000000000000) at ffmpeg_filter.c:2317:16
>>> frame #6: 0x0000005555bc7e48 ffmpeg_g`filter_thread(arg=0xb400007eae2ce7a0) at ffmpeg_filter.c:2836:15
>>> frame #7: 0x0000005555bee568 ffmpeg_g`task_wrapper(arg=0xb400007d8e2db478) at ffmpeg_sched.c:2200:21
>>> 
>>> Signed-off-by: Zhao Zhili <zhilizhao at tencent.com>
>>> ---
>>> This bug is trigged by 10bit H.264 which doesn't supported by mediacodec.
>>> There is some misleading error messages from ffmpeg cli:
>>> 
>>> [h264_mediacodec @ 0xb400007513788010] Failed to dequeue output buffer (status=-10000)
>>> [vist#0:0/h264 @ 0xb4000074a3786b30] Error submitting packet to decoder: Generic error in an external library
>>> [h264_mediacodec @ 0xb400007513788010] Failed to dequeue output buffer (status=-10000)
>>> [vist#0:0/h264 @ 0xb4000074a3786b30] Error submitting packet to decoder: Generic error in an external library
>>> [vist#0:0/h264 @ 0xb4000074a3786b30] A decoder returned an unexpected error code. This is a bug, please report it.
>>> [vist#0:0/h264 @ 0xb4000074a3786b30] Error processing packet in decoder: Internal bug, should not have happened
>>> 
>>> fftools/ffmpeg_filter.c | 2 +-
>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>> 
>>> diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c
>>> index bb755d7bb4..9dc47f9d90 100644
>>> --- a/fftools/ffmpeg_filter.c
>>> +++ b/fftools/ffmpeg_filter.c
>>> @@ -2192,7 +2192,7 @@ static int close_output(OutputFilterPriv *ofp, FilterGraphThread *fgt)
>>> 
>>>    // we are finished and no frames were ever seen at this output,
>>>    // at least initialize the encoder with a dummy frame
>>> -    if (!fgt->got_frame) {
>>> +    if (!fgt->got_frame && ofp->format != -1) {
>> 
>> I don't quite understand how precisely does this happen.
>> 
>> This code should only be reachable if the filtergraph was configured at
>> least once, then configure_filtergraph() should set ofp->format to the
>> format reported by the lavfi buffersink. Does this then mean that lavfi
>> is configured with AV_PIX_FMT_NONE? Or that this is somehow triggered
>> without the filtergraph being configured?
> 
> 
> Can reproduced with:
> 
> diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c
> index 559f63698a..af4ea37812 100644
> --- a/libavcodec/qsvdec.c
> +++ b/libavcodec/qsvdec.c
> @@ -1050,6 +1050,7 @@ static int qsv_decode_frame(AVCodecContext *avctx, AVFrame *frame,
>     QSVDecContext *s = avctx->priv_data;
>     int ret;
> 
> +    return AVERROR_EXTERNAL;
>     /* buffer the input packet */
>     if (avpkt->size) {
>         AVPacket input_ref;
> 
> Then
> ./ffmpeg -c:v h264_qsv -xerror  -I input.mp4 -c:a copy  -c:v libx264  output.mp4 
> 
> Looks like filtergraph isn’t being configured.

By the way, this issue is related to trac 
https://trac.ffmpeg.org/ticket/10671

User reported different behavior on 6.1, like does exit with -xerror but continue with audio stream.

> 
>> 
>> -- 
>> Anton Khirnov
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel at ffmpeg.org <mailto:ffmpeg-devel at ffmpeg.org> <mailto:ffmpeg-devel at ffmpeg.org>
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> 
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-request at ffmpeg.org <mailto:ffmpeg-devel-request at ffmpeg.org> <mailto:ffmpeg-devel-request at ffmpeg.org> with subject "unsubscribe".
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org <mailto:ffmpeg-devel at ffmpeg.org>
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org <mailto:ffmpeg-devel-request at ffmpeg.org> with subject "unsubscribe".