[FFmpeg-devel] [PATCH] mov: fix decode of fragments that overlap in time

[John Stebbins]

On 09/26/2017 05:12 PM, Michael Niedermayer wrote:
> Hi
>
> On Mon, Sep 25, 2017 at 02:12:26PM -0700, John Stebbins wrote:
>>
>> On 09/25/2017 01:12 PM, Carl Eugen Hoyos wrote:
>>> 2017-09-25 19:10 GMT+02:00 John Stebbins <jstebbins at jetheaddev.com>:
>>>> When keyframe intervals of dash segments are not perfectly aligned,
>>>> fragments in the stream can overlap in time. Append new "trun" index
>>>> entries to the end of the index instead of sorting by timestamp.
>>>> Sorting by timestamp causes packets to be read out of decode order and
>>>> results in decode errors.
>>>> ---
>>>>   libavformat/mov.c | 4 ++--
>>>>   1 file changed, 2 insertions(+), 2 deletions(-)
>>>>
>>>> diff --git a/libavformat/mov.c b/libavformat/mov.c
>>>> index 2519707345..b2bc7c2c3d 100644
>>>> --- a/libavformat/mov.c
>>>> +++ b/libavformat/mov.c
>>>> @@ -4339,8 +4339,8 @@ static int mov_read_trun(MOVContext *c, AVIOContext *pb, MOVAtom atom)
>>>>                                     MOV_FRAG_SAMPLE_FLAG_DEPENDS_YES));
>>>>           if (keyframe)
>>>>               distance = 0;
>>>> -        ctts_index = av_add_index_entry(st, offset, dts, sample_size, distance,
>>>> -                                        keyframe ? AVINDEX_KEYFRAME : 0);
>>>> +        ctts_index = add_index_entry(st, offset, dts, sample_size, distance,
>>>> +                                     keyframe ? AVINDEX_KEYFRAME : 0);
>>> I can confirm that this fixes playback with FFplay but it shows
>>> many warnings (Non-monotonous DTS) with ffmpeg: Is this
>>> unavoidable?
>>>
>>>
>> Fixing the non-monotonous DTS in ffmpeg would require more consideration. The overlapping frames need to be dropped
>> somewhere after they are decoded and before they are presented.
>>
>> I've given this some thought, but other ideas are certainly welcome. The demuxer is probably the most appropriate place
>> for marking the frames as needing to be dropped since it has the "knowledge" about why there are overlapping
>> timestamps.  I.e. you only want to drop frames in this particular scenario and not generally when timestamps go
>> backwards.  I don't think there is currently any facility for marking frames to be dropped after decoding, but it seems
>> like AVPacketSideData would be the appropriate mechanism for such marking.  FYI, such a facility for marking frames to
>> drop would also be useful for frame accurate playback of MP4 edit lists.
> This sounds like:
>
> /**
>   * Flag is used to discard packets which are required to maintain valid
>   * decoder state but are not required for output and should be dropped
>   * after decoding.
>   **/
> #define AV_PKT_FLAG_DISCARD   0x0004
>
>
>

Oh, that's awesome. Thanks for pointing that out. There's even a sample 
flag that can be set in the index which is the next thing I was 
"worried" about.  I'll follow up with a patch to fix the non-monotonous 
DTS.