[FFmpeg-devel] [PATCH] dashenc: Write out DASH manifest immediately in streaming mode

[Jeyapal, Karthick]

On 6/11/21, 9:31 PM, "Kevin LaFlamme" <kevin at aiera.com> wrote:

>Does my last explanation make sense or if not could you point me to
>where this reasoning is incorrect?
Your reasoning is perfectly fine. Since I haven't heard any objections to your reply, I have pushed this patch. Thanks.

>
>Kevin LaFlamme
>Director of Engineering (Front End)
>774.265.0382 (m)
>aiera.com
>On Jun 8, 2021, 8:52 PM -0400, Kevin LaFlamme <kevin at aiera.com>, wrote:
>> I just realized there’s probably a more straightforward explanation:
>>
>> The scenario you brought up is already happening for all segments after
>> the first one, and this changes just makes the first one do the same
>> thing.
>>
>> The following scenario happens even without this change:
>>
>> Say you have a stream with "-streaming" and "-use_template" set and a
>> segment duration of 10 seconds. While the first segment is being
>> written there is no manifest, but then the first segment and manifest
>> are written and the segment segment starts to be written. At 15 seconds
>> in a client requests the manifest. It wants to be 2 seconds behind the
>> live edge (13 second seek position) so it calculates which segment name
>> it needs, which is the second one, and makes a request for it even
>> though it's only partially written. And this could currently happen for
>> any segment index > 1.
>>
>> With the proposed change this same thing just now happens on the first
>> segment as well.
>>
>> I'm definitely happy to try a different approach to fix the underlying
>> issue if I'm misunderstanding something or this reasoning is incorrect,
>> but this "seems" right to me from my understanding of the spec and the
>> behaviors I've witnessed so far.
>>
>> Kevin LaFlamme
>> Director of Engineering (Front End)
>> 774.265.0382 (m)
>> aiera.com
>> On Jun 8, 2021, 4:03 PM -0400, Kevin LaFlamme <kevin at aiera.com>, wrote:
>> > To serve it in a truly streaming way does require some special purpose server or configuration, but >the `@availabilityTimeComplete` field that is written into the manifest is supposed to indicate to the >client making the request whether or not the segment file is complete at the time it becomes available. >In streaming mode, this is set to false so the client should expect that it can get a partial segment >between the available start time and the segment duration.
>> >
>> > So as each box is written to the file, it should be safe for the HTTP server to send the partial >segment file that is currently on disk. And the DASH spec specifically indicates whether or not to expect >a partial file for the segment being requested. If the client can’t handle a partial file and is still making >an attempt to request it before it’s fully available (which is calculable from the information in the >manifest), the client wouldn’t be respecting the DASH spec.
>> >
>> > Kevin LaFlamme
>> > Director of Engineering (Front End)
>> > 774.265.0382 (m)
>> > aiera.com
>> > On Jun 8, 2021, 3:34 PM -0400, Timo Rothenpieler <timo at rothenpieler.org>, wrote:
>> > > On 08.06.2021 21:24, Kevin LaFlamme wrote:
>> > > > For streaming mode with fragmented MP4 the intention is to have the client read a partial file >since it’s broken up into sequential boxes that are playable independently. This doesn’t change the >segment files themselves or how they are written, it just writes a full index file ahead of time.
>> > > >
>> > > > Even currently, the manifest gets written after the first segment is written and the player can >immediately attempt to start reading the second segment before it’s fully written.
>> > > >
>> > > > This is the same thing that the “-lhls” is doing in the block below, writing out the HLS manifest >immediately so that it contains X-EXT-PREFETCH fields for the partial segment files.
>> > > >
>> > > > Currently, if you have a low latency DASH stream with “-ldash” and “-streaming” and the player >joins in the middle of the stream everything works and it starts playing from the middle of the current >segment. This means even with 10 second segments you can have latency < 1sec of the live edge. >However, this doesn’t work for the very first segment because the manifest isn’t available to the player >yet.
>> > > >
>> > > > I have a low-latency player where I’m seeing this issue right now and this addresses the >problem, and seems consistent with the “-lhls” handling below, but happy to make changes if there is >something else I’m missing.
>> > >
>> > > Yes, I'm aware that that's how it's supposed to work in theory.
>> > > But does that work with any HTTP server?
>> > > Do they really manage to stream the file while it's being written to?
This feature works with most of the popular CDNs. And yes, they do manage a file as it is being written into.

>> > > Or does this need special software to serve the file? And in turn, will
>> > > it break for a user just pointing apache/nginx at such a list?

To start with, -streaming feature in dashenc, will not work with a standard apache/nginx.
This feature needs a http server that handles a streaming.
This change is just a bugfix to -streaming feature. 

regards,
Karthick
>> > >
>> > >
>> > > _______________________________________________
>> > > ffmpeg-devel mailing list
>> > > ffmpeg-devel at ffmpeg.org
>> > >
>> > > To unsubscribe, visit link above, or email
>> > > ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".