[FFmpeg-devel] [PATCH] fftools/ffmpeg_filter: at EOF, unref frame used for FPS conversion logic

[Steven Zhou]

> >>> On Jun 24, 2025, at 11:09, Steven Zhou <steven.zhou at netint.ca> wrote:
> >>>
> >>> fftools/ffmpeg_filter: at EOF, unref frame used for FPS conversion
> >>> logic
> >>>
> >>> Ever since commit 64f3feb added
> >>> `avcodec_free_context(&enc->enc_ctx);`
> >>> to `enc_free()` the encoder context is closed before filtergraph
> >>> output
> >> context.
> >>> The filtergraph output context holds a reference to last AVFrame to
> >>> pass through it for FPS conversion purposes. This is problematic for
> >>> HWFrameContexts which may rely on encoder context to free HW
> >> AVFrames.
> >>
> >> What's the encoder? This sounds a defect in encoder implementation.
> >
> > The encoder is the yet to be upstreamed Netint encoders. I will put them on
> Forgejo for review later this week.
> > Perhaps the encoder implementation does not conform to FFmpeg
> convention. If so, I can try to address this issue another way.
> >
> > The Netint HW frame data held in `AVFrame->buf[0]` is tied 'sessions' of
> Netint HW dec/filt/enc in the Netint driver based on what Netint HW session
> most recently accessed it.
> > The AVBufferRef destructor for `AVFrame->buf[0]` in Netint HW frames will
> use the most recent session the HW frame passed through to communicate its
> destruction to the HW's driver.
> > This allows for the decoder session which generated a Netint HW AVframe to
> have its context and session closed whilst the AVFrame continues onto a
> Netint encoder.
> > Typically this occurs when decoder is closed for EOF whilst frames are still in
> flight to the encoder.
> 
> Both v4l2_m2m and mediacodec decoders have similar issue, and they use
> reference count to delay destroy decoder context when there are frames on
> the fly, e.g., mediacodec_decode_close. It's a little complex, but more reliable
> than fix on the user side of libavcodec.
> 
Ok, thanks for the advice. I'll take a look at those hw accelerators and try to
fix the netint encoder issue without changing ffmpeg framework.
This patch can be cancelled.
> >
> > I noticed commit 64f3feb closes encoder before filtergraph; but filtergraph
> holds a ref to last AVFrame for FPS conversion usage.
> > This ref in filtergraph could be discarded earlier when filtergraph processes
> the EOF frame, hence this patch.
> >
> > Alternatively, the Netint codecs can do without this patch. The frame's
> resources on HW would be free'd by other garbage-collection mechanisms.
> >
> >>
> >>> This change unrefs the AVFrame
> >>> ref held by filtergraph output context during processing of EOF frame.
> >>>
> >>> Signed-off-by: Steven Zhou <steven.zhou at netint.ca>
> >>> ---
> >>> fftools/ffmpeg_filter.c | 4 +++-
> >>> 1 file changed, 3 insertions(+), 1 deletion(-)
> >>>
> >>> diff --git a/fftools/ffmpeg_filter.c b/fftools/ffmpeg_filter.c index
> >>> a0dc4c745e..cf2d016b2d 100644
> >>> --- a/fftools/ffmpeg_filter.c
> >>> +++ b/fftools/ffmpeg_filter.c
> >>> @@ -2560,8 +2560,10 @@ static int fg_output_frame(OutputFilterPriv
> >> *ofp, FilterGraphThread *fgt,
> >>>        av_frame_move_ref(frame_prev, frame);
> >>>    }
> >>>
> >>> -    if (!frame)
> >>> +    if (!frame) {
> >>> +        av_frame_unref(frame_prev);
> >>>        return close_output(ofp, fgt);
> >>> +    }
> >>>
> >>>    return 0;
> >>> }
> >>> --
> >>> 2.25.1
> >>>