[FFmpeg-devel] [PATCH] fftools/ffmpeg_filter: at EOF, unref frame used for FPS conversion logic

[Zhao Zhili]

> On Jun 24, 2025, at 12:47, Steven Zhou <steven.zhou at netint.ca> wrote:
> 
> 
> 
>> -----Original Message-----
>> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of Zhao
>> Zhili
>> Sent: Monday, June 23, 2025 8:34 PM
>> To: FFmpeg development discussions and patches <ffmpeg-
>> devel at ffmpeg.org>
>> Subject: Re: [FFmpeg-devel] [PATCH] fftools/ffmpeg_filter: at EOF, unref frame
>> used for FPS conversion logic
>> 
>> 
>> 
>>> On Jun 24, 2025, at 11:09, Steven Zhou <steven.zhou at netint.ca> wrote:
>>> 
>>> fftools/ffmpeg_filter: at EOF, unref frame used for FPS conversion
>>> logic
>>> 
>>> Ever since commit 64f3feb added `avcodec_free_context(&enc->enc_ctx);`
>>> to `enc_free()` the encoder context is closed before filtergraph output
>> context.
>>> The filtergraph output context holds a reference to last AVFrame to
>>> pass through it for FPS conversion purposes. This is problematic for
>>> HWFrameContexts which may rely on encoder context to free HW
>> AVFrames.
>> 
>> What’s the encoder? This sounds a defect in encoder implementation.
> 
> The encoder is the yet to be upstreamed Netint encoders. I will put them on Forgejo for review later this week.
> Perhaps the encoder implementation does not conform to FFmpeg convention. If so, I can try to address this issue another way.
> 
> The Netint HW frame data held in `AVFrame->buf[0]` is tied 'sessions' of Netint HW dec/filt/enc in the Netint driver based on what Netint HW session most recently accessed it.
> The AVBufferRef destructor for `AVFrame->buf[0]` in Netint HW frames will use the most recent session the HW frame passed through to communicate its destruction to the HW's driver.
> This allows for the decoder session which generated a Netint HW AVframe to have its context and session closed whilst the AVFrame continues onto a Netint encoder.
> Typically this occurs when decoder is closed for EOF whilst frames are still in flight to the encoder.

Both v4l2_m2m and mediacodec decoders have similar issue, and they use reference count to delay destroy
decoder context when there are frames on the fly, e.g., mediacodec_decode_close. It’s a little complex, but more
reliable than fix on the user side of libavcodec.

> 
> I noticed commit 64f3feb closes encoder before filtergraph; but filtergraph holds a ref to last AVFrame for FPS conversion usage.
> This ref in filtergraph could be discarded earlier when filtergraph processes the EOF frame, hence this patch.
> 
> Alternatively, the Netint codecs can do without this patch. The frame's resources on HW would be free'd by other garbage-collection mechanisms.
> 
>> 
>>> This change unrefs the AVFrame
>>> ref held by filtergraph output context during processing of EOF frame.
>>> 
>>> Signed-off-by: Steven Zhou <steven.zhou at netint.ca>
>>> ---
>>> fftools/ffmpeg_filter.c | 4 +++-
>>> 1 file changed, 3 insertions(+), 1 deletion(-)
>>> 
>>> diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index
>>> a0dc4c745e..cf2d016b2d 100644
>>> --- a/fftools/ffmpeg_filter.c
>>> +++ b/fftools/ffmpeg_filter.c
>>> @@ -2560,8 +2560,10 @@ static int fg_output_frame(OutputFilterPriv
>> *ofp, FilterGraphThread *fgt,
>>>        av_frame_move_ref(frame_prev, frame);
>>>    }
>>> 
>>> -    if (!frame)
>>> +    if (!frame) {
>>> +        av_frame_unref(frame_prev);
>>>        return close_output(ofp, fgt);
>>> +    }
>>> 
>>>    return 0;
>>> }
>>> --
>>> 2.25.1
>>> 
>>> _______________________________________________
>>> ffmpeg-devel mailing list
>>> ffmpeg-devel at ffmpeg.org
>>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>> 
>>> To unsubscribe, visit link above, or email
>>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>> 
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel at ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> 
>> To unsubscribe, visit link above, or email ffmpeg-devel-request at ffmpeg.org
>> with subject "unsubscribe".
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".