[FFmpeg-user] ffmpeg retry failed segments

[Torsten Kaiser]

Since the freeze seems to appear only with the first entry in the 
playlist, within the first download, you also could try to "plow the 
road" before actually trying to download via ffmpeg.

e.g. something like this:

wget --spider <m3u8-URL>

sleep 1

ffmpeg <your parameters>

In spider-mode wget doesn't actually download the playlist, but tries to 
reach it prior to your invocation of ffmpeg. This way the route across 
the web is already scouted before ffmpeg tries to reach for the first 
input file in the list

Honestly, I don't really expect ffmpeg to have a parameter to wait for 
the outside world, since the "ff" in ffmpeg means "fast forward"

Good luck!


On 27.09.23 13:48, Stein Rune Risa wrote:
> Ok - I think I will investigate the output from ffmpeg for some of the
> failed cases to see what happens. But ffmpeg does somehow skip some
> segments, but next time they might be OK.
>
> I also learnt that I can use xerror parameter to stop if it encounters
> errors. Then it is easy to detect it without visually inspect it.
>
>
>
> On Wed, Sep 27, 2023 at 1:44 PM Peter van den Houten <petervdh at gmail.com>
> wrote:
>
>> Hi, reply below.
>>
>> On 27/09/2023 11:03, Torsten Kaiser wrote:
>>> Waiting longer might not be helping. What if the first entry is indeed
>> up to 5 seconds long, then immediately switching the source to the next
>> URL? New URL, new TLS-handshake. Browsers do indeed wait longer and retry
>> multiple times to get a lock to the next bit of any web content. Visible to
>> the user by the "hour glass".
>>> But: Knowing the location of the source m3u8 could provide the
>> opportunity to download the playlist itself via [wget <m3u8-url>]. The
>> following link to Wikipedia describes the structure of m3u-files.
>>> https://en.wikipedia.org/wiki/M3U
>>>
>>> So it's all plain text. Media entries are local relative links or URLs,
>> so you may want to keep the m3u8's location
>>> Replacing the m3u8-filename with one of the media entries in question
>> (if local append whole path, otherwise replace the whole construct with the
>> URL-entry) should be a valid URL to the media source, provided the issuer
>> allows access without evaluating the http-referer as being of its own realm.
>>> So prepared, try to use ffmpeg with this single media file as input url
>> , watching for errors.
>>> No solution, but trying to circumvent suspected problems with certain
>> entries in the playlist, narrowing down the problem
>>> (And learned for myself what hides behind HLS streaming)
>>>
>>> On 27.09.23 09:00, Stein Rune Risa wrote:
>>>> Hi!
>>>>
>>>> This is a very stable and well known source, so the network is also
>> stable.
>>>> There are never any issues streaming through their official player.
>>>>
>>>> I also see that the streams are delivered through well known CDN - so it
>>>> should be fine.
>>>>
>>>> But I will try the xerror option. It is better to detect errors than
>>>> getting some problem with the file.
>>>>
>>>> It also seems that the hiccup always happens within the first 5 seconds
>> of
>>>> the stream - so there might be some "initial delay" when initializing
>> the
>>>> stream. Is there some parameter for ffmpeg to kind of allow longer
>> response
>>>> times from the source?
>>>>
>>>>
>>>>
>>>> On Wed, Sep 27, 2023 at 8:16 AM Torsten Kaiser <info at linux-rebellen.de>
>>>> wrote:
>>>>
>>>>> Looks like you download a known playlist from some unstable network
>>>>> connection.
>>>>>
>>>>> To get a grasp on this: If you set the -xerror switch like
>>>>>
>>>>> ffmpeg -y -xerror -i https://file.com/myfiles.m3u8 -c copy
>> /tmp/test.mkv
>>>>> what happens? Download should stop at the first buggy/missing segment
>> of
>>>>> the playlist. Now repeat the download multiple times to see if the
>>>>> problem is random or not.
>>>>>
>>>>> Missing segments could be a DRM problem or invalid/outdated entries in
>>>>> the playlist. If the download always misses the same segment it's most
>>>>> likely the segment entry is invalid.
>>>>>
>>>>> Buggy segments are most likely either already buggy in the source or a
>>>>> network bottleneck. If the faults always occur on the same spot, then
>>>>> the source itself is most likely already buggy. Encountered this myself
>>>>> on legacy sources digitized from VHS tapes. Sometimes content is more
>>>>> important than quality.
>>>>>
>>>>>
>>>>> On 26.09.23 22:36, Stein Rune Risa wrote:
>>>>>> I am using ffmpeg to download some streaming to a mkv file. The
>> stream parts comes in a m3u8 file.
>>>>>> ffmpeg -y -i https://file.com/myfiles.m3u8 -c copy /tmp/test.mkv
>>>>>>
>>>>>> In some cases, there have been some issues downloading some of the
>> segments, and the mkv files have parts where it freezes for some seconds
>> before continuing. I am now forced to check the file manually (by watching)
>> and then redownload if it has any issues.
>>>>>> I guess this happens because ffmpeg temporarily is not able to
>> download some of the segments referred to in the m3u8?
>>>>>> Is there some way that I can force ffmpeg to retry failed segments?
>> Or at least abort if it happens? I am kind of looking for a way to get rid
>> of themanual checking of the content.
>>
>>
>> As Torsten pointed out, the problem could be DRM or outdated/unavailable
>> segments. I have had no problems using the following code, with a user
>> agent string which keeps the website happy and downloads all segments from
>> most websites, maybe it will help with your instance. This example is to
>> download a copyright-free movie from PlexVideos:
>>
>> ffmpeg -user_agent "Mozilla/5.0 (iPad; CPU OS 6_0 like Mac OS X)
>> AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A5355d
>> Safari/8536.25" -i "
>> https://vod-content.plexvideos.com/streams/62be17835e5872e950b87398/0f59c0e3-59ad-4362-8cf2-11f8508c46f2/stream_6.m3u8"
>> -c copy -y the_brain_that_wouldn't_die.mkv.
>>
>> _______________________________________________
>> ffmpeg-user mailing list
>> ffmpeg-user at ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>>
>> To unsubscribe, visit link above, or email
>> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
>>
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".