[FFmpeg-devel] FIX: HLS Discontinuity / Non-Monotonous DTS

[Joe Koberg]

I'm saying I don't think it needs it - MPEGTS does seem to handle the
discontinuous input fine if I just concatenate it.

MPEG TS packets themselves do have a discontinuity flag in the
adaptation field, but again it looks like it might be working as-is
without explicit notification of them. (and probably should)

The question remains: what is missing to allow the HLS format to
behave as MPEG-TS does right now?



On Fri, Mar 16, 2018 at 3:36 PM, Bodecs Bela <bodecsb at vivanet.hu> wrote:
>
>
> 2018.03.16. 19:52 keltezéssel, wm4 írta:
>>
>> On Fri, 16 Mar 2018 09:58:17 -0400
>> Joe Koberg <joe at koberg.us> wrote:
>>
>>> Hello,
>>>
>>> This patch adds code to track and correct timestamp discontinuities,
>>> fixing
>>> "non-monotonous dts" errors and timing issues with HLS playlists.
>>>
>>> For some time, FFmpeg has not properly handled discontinuous timestamps
>>> in
>>> the MPEG-TS stream from Apple HLS playlists. Symptoms include inability
>>> to
>>> remux to containers like MP4 or stream to RTMP endpoints; choppy, broken,
>>> or sped-up playback; and voluminous "Non-monotonous DTS", "went
>>> backwards",
>>> and "out of order" messages logged.  This is particularly troublesome in
>>> the context of HLS, as the format allows insertion of
>>> independently-sourced
>>> fragments for ad-insertion or to concatenate ongoing video to a stream.
>>>
>>> Over the years, many users have encountered and reported this issue in
>>> different ways.
>>>
>>>    * https://trac.ffmpeg.org/ticket/5419 "HLS EXT-X-DISCONTINUITY tag is
>>> not
>>> supported"
>>>    * https://trac.ffmpeg.org/ticket/6613
>>>    * https://trac.ffmpeg.org/ticket/6810
>>>    * https://trac.ffmpeg.org/ticket/6709
>>>    * https://trac.ffmpeg.org/ticket/6365
>>>    * https://trac.ffmpeg.org/ticket/5236
>>>    * https://github.com/Bilibili/ijkplayer/issues/2565
>>>    *
>>> https://stackoverflow.com/questions/49289394/downloading-ts-stream-with-
>>> ext-x-discontinuity-sequence-ffmpeg
>>>    * etc...
>>>
>>>
>>> It appears that the -dts_delta_threshold checking code is quite similar
>>> in
>>> intent, and adjacent to this code, but the HLS demuxer lacks the
>>> AVFMT_TS_DISCONT flag, effectively disabling that option.  Even when
>>> AVFMT_TS_DISCONT is set on the input format (e.g. MPEG-TS),
>>> "-dts_delta_threshold 0" produces out-of-sync audio in our tests.  That
>>> code has been left as-is for now.  The flag probably should be set on the
>>> HLS format, though.
>>>
>>> In this fix, any backwards DTS jump is considered an error, as is any
>>> forward jump of DTS or PTS by more than `dts_monotonicity_threshold`
>>> microseconds (really, AV_TIME_BASE units, defaulting to 1000000 = 1
>>> second). The delta from the predicted value is updated and applied to all
>>> subsequent packets, with the assumption that video was stitched together
>>> from continuous runs.
>>>
>>> With this patch in place and the -force_dts_monotonicity flag set, we can
>>> successfuly transcode arbitrary HLS playlists to MP4, FLV, or other
>>> timestamped containers when we could not before.  The playback
>>> presentation
>>> of the result is smooth, properly timed, and in-sync in Quicktime Player,
>>> VLC, and ffplay with correctly reported and the file seekable.
>>>
>>> Interaction with flags such as -copyts, -copytb, and others has not been
>>> fully analyzed.
>>>
>>> We have prepared a test HLS playlist that reveals the issue.
>>>
>>> How to reproduce:
>>>
>>>    ffmpeg -i https://s3.amazonaws.com/playon-test-videos/discont_
>>> test_new/discont_test.m3u8 -c copy -y output.mp4
>>>
>>> With patch:
>>>
>>>    ffmpeg -force_dts_monotonicity -i https://s3.amazonaws.com/
>>> playon-test-videos/discont_test_new/discont_test.m3u8 -c copy -y
>>> output.mp4
>>>
>>>
>>> Please let me know if I've missed anything!
>>
>> So I haven't looked too closely at this. It's possible that ffmpeg.c
>> should handle discontinuities better, or has a regression with handling
>> it. But besides that, shouldn't the HLS demuxer just be changed to
>> parse the discontinuity markers, and add them as packet flags? Then
>> these "clever" heuristics (which are just really hard to maintain)
>> wouldn't be needed, and also applications using libavformat as library
>> would benefit from it.
>>
>> Your patch may or may not have a benefit on top of it (for normal .ts
>> files etc.), and the HLS demuxer may or may not have additional issues,
>> but I think that's orthogonal.
>>
>> There isn't such a discontinuity packet flag - that would have to be
>> added.
>
> maybe this  discontinuity packet flag could be used in concat demuxer also?
>>
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel at ffmpeg.org
>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> bb
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel