[FFmpeg-devel] [PATCH] [RFC] libavcodec/hevc_refs: Clear DPB of old frames

[Kieran Kunhya]

On Mon, 3 Aug 2020 at 09:18, Anton Khirnov <anton at khirnov.net> wrote:

> Quoting Kieran Kunhya (2020-07-16 21:32:34)
> > During glitching or looping streams, old frames remain in the DPB.
> > The decoder incorrectly thinks that the DPB contains the right number of
> > buffered frames to output and reordering breaks badly
> >
> > Any non-cosmetic comments welcome.
> >From 23f0272092f6528a268abe7e6a7fd8764553048f Mon Sep 17 00:00:00 2001
> >From: Kieran Kunhya <kierank at obe.tv>
> >Date: Thu, 16 Jul 2020 20:29:24 +0100
> >Subject: [PATCH] [RFC] libavcodec/hevc_refs: Clear DPB of old frames
> >
> >During glitching or looping streams, old frames remain in the DPB.
> >The decoder incorrectly thinks that the DPB contains the right number of
> buffered frames to output and reordering breaks badly
>
> Can you describe what happens in more details? It's not obvious what you
> "breaks badly" means. Even if some frame gets skipped, the old frames
> should still get output in the correct order, since their POCs are
> correct.
>

Here is my writeup:

The problem is that after the wrap around the DPB is not cleared correctly:

"DPB List: idx 0 poc -35588, idx 1 poc -7779, idx 2 poc -7772, idx 3 poc
-7768, idx 4 poc -7776, idx 5 poc -7780, idx 6 poc -7775, idx 7 poc -7777,
idx 8 poc -7781, idx 9 poc -7778, idx 10 poc -7774, idx 11 poc -7773, idx
12 poc -35592, idx 13 poc -35596, idx 14 poc -35599, idx 15 poc -35597, idx
16 poc -35601, idx 17 poc 0, idx 18 poc 0, idx 19 poc 0, idx 20 poc 0, idx
21 poc 0, idx 22 poc 0, idx 23 poc 0, idx 24 poc 0, idx 25 poc 0, idx 26
poc 0, idx 27 poc 0, idx 28 poc 0, idx 29 poc 0, idx 30 poc 0, idx 31 poc
0,"

The ~7000 series is the previous set of frames and after the wrap around
it's ~35000.
The logic here makes sure there is enough frames in the DPB before
outputting:
https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/hevc_refs.c#L203

But it gets confused because of the old frames in the DPB and thinks it's
ready to output. The logs show this as follows:

verbose: [102828010902015] [av] Output frame with POC -35588 dpb-idx 0.
verbose: [102828011180970] [av] Decoded frame with POC -35588.

This is wrong because the frame is decoded then immediately output so there
is no reordering that takes place.

During normal decode it looks more like this:

verbose: [102824827016306] [av] Output frame with POC -14866 dpb-idx 0.
verbose: [102824827130789] [av] Decoded frame with POC -14857.

(i.e there is reordering)


> >---
> > libavcodec/hevc_refs.c | 19 +++++++++++++++++++
> > 1 file changed, 19 insertions(+)
> >
> >diff --git a/libavcodec/hevc_refs.c b/libavcodec/hevc_refs.c
> >index 4f6d985..a8e0028 100644
> >--- a/libavcodec/hevc_refs.c
> >+++ b/libavcodec/hevc_refs.c
> >@@ -277,6 +277,10 @@ static int init_slice_rpl(HEVCContext *s)
> >     int ctb_addr_ts  =
> s->ps.pps->ctb_addr_rs_to_ts[s->sh.slice_segment_addr];
> >     int i;
> >
> >+    if(frame && !frame->rpl_buf) {
>
> How can this happen? From looking at the code, I don't see how either of
> those conditions can possibly be true.
>

>From memory I had segfaults if that check was not there.

Kieran