[FFmpeg-devel] [PATCH] Add Apple HTTP Live Streaming protocol handler

[Martin Storsjö]

On Mon, 26 Jul 2010, Ronald S. Bultje wrote:

> Hi,
> 
> On Mon, Jul 26, 2010 at 1:36 PM, Martin Storsj? <martin at martin.st> wrote:
> > On Mon, 26 Jul 2010, Ronald S. Bultje wrote:
> >> On Mon, Jul 26, 2010 at 12:29 PM, Martin Storsj? <martin at martin.st> wrote:
> >> > On Mon, 26 Jul 2010, Ronald S. Bultje wrote:
> >> >> It goes further than backing up. As we know, TCP is stateless. We can
> >> >> download from a URI, transfer to a different WIFI network and the
> >> >> download will in fact resume (this can be at TCP-level or at HTTP
> >> >> level using the offset attribute - in this case I mean at the
> >> >> TCP-level). Imagine that I'm using "ffmpeg" (...) on a cellphone and I
> >> >> switch my iPhone from WiFi to 3G because I'm driving my car into
> >> >> no-man's-land, I'd like my streaming to continue, just dynamically
> >> >> switch to a different bitrate.
> >> >>
> >> >> For RDT/Realmedia streams (and any local file), this works (press 'a',
> >> >> 'v' or 's' to switch audio/video/subtitle stream in ffplay). An
> >> >> intelligent application on my iPhone could do this dynamically
> >> >> depending on my current network settings (3G? edge? WiFi?) and/or
> >> >> bandwidth (as determined by whatever mechanism). This only works if
> >> >> such info is presented to the application.
> >> >
> >> > Yes, and that can be implemented just as well on the URLProtocol level,
> >> > automatically sensing the available bandwidth and switching to another
> >> > variant playlist. The data fed to the demuxer simply suddenly is from
> >> > another stream with lower bitrate, but the specs for this protocol
> >> > mandates that the variant streams are similar enough so that this works.
> >> >
> >> > Only if you want to manually control which bitrate variant is chosen on
> >> > the fly would you need some more control over the receiving than what you
> >> > can have with URLProtocols at the moment. But I don't think a demuxer is
> >> > any better suited for it either - the bandwidth info is much better known
> >> > at the URLProtocol level than forcing the application itself try to guess
> >> > it and switch variants by adjusting discard flags for different demuxer
> >> > streams.
> >>
> >> I totally agree. Both demuxers and protocols should stay off of this
> >> stuff. It's the application's job to do this. So we need to expose
> >> that this is possible to the application. Realmedia/RDT (or
> >> .rm/.avi/.matroska files that have multiple tracks, or DVDs with
> >> multiple soundtracks) show what API we use for that. I'd like you to
> >> use this API also, even if only for consistency.
> >
> > Ok, I can give it a try. Doing it that way makes some things prettier, but
> > some other things get a bit uglier.
> >
> > In this setup, we'd need to convey info about which streams belong to
> > which bandwidth variant, since (especially in low bandwidth setups), you
> > wouldn't want to pull video from one stream and audio from another one,
> > forcing you to receive both variants.
> 
> "Coupling", I guess. Yes, it makes sense to add some mechanism. *If*
> the application is silly enough to request that, it should work, but
> it makes sense to not make it the default, so a coupling ID of some
> sort would make sense to me.

Here's the initial attempt. While this has the upside that it gives you 
much more flexibility, the code is also vastly more complicated than the 
earlier attempt and a bit longer (>500 lines compared to 300).

Another downside is that it is much harder to implement automatic quality 
switching on the libavformat level (if that ever was desired) in this 
setup - but of course allows much better manual control.

I expose the variant number that each stream belongs to in AVStream->id - 
I hope that is ok, it's quite useful for mapping the streams to the 
internal variants internally, too.

Unfortunately enough, ffplay picks the stream to use using the 
AVStream->codec_info_nb_frames value, which makes it more often than not 
pick audio and video streams from different variants. If starting with 
-vst 0 -ast 0, it uses the first of each, which should belong to the same 
variant, and if switching by pressing 'a' and 'v', pressing both at the 
same time to keep taking both variants from the same streams is 
recommended.

Also, with the sample stream at 
http://devimages.apple.com/iphone/samples/bipbop/bipbopall.m3u8 at least, 
ffplay seems to be buffering way too much data, so when switching streams, 
we jump very far ahead. For testing with this stream, it can be fixed with 
a modification like this:

--- a/ffplay.c
+++ b/ffplay.c
@@ -2546,7 +2546,7 @@ static int decode_thread(void *arg)
 
         /* if the queue are full, no need to read more */
         if (   is->audioq.size + is->videoq.size + is->subtitleq.size > MAX_QUEUE_SIZE
-            || (   (is->audioq   .size  > MIN_AUDIOQ_SIZE || is->audio_stream<0)
+            || (   (is->audioq   .nb_packets > MIN_FRAMES || is->audio_stream<0)
                 && (is->videoq   .nb_packets > MIN_FRAMES || is->video_stream<0)
                 && (is->subtitleq.nb_packets > MIN_FRAMES || is->subtitle_stream<0))) {
             /* wait 10 ms */

Note, I'm not really proposing this ffplay patch in general, but I had to 
do this modification for stream switching to work well.

// Martin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Make-parse_key_value-from-httpauth-a-common-lavf-int.patch
Type: text/x-diff
Size: 8108 bytes
Desc: 
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20100727/078cc453/attachment.patch>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-Add-Apple-HTTP-Live-Streaming-demuxer.patch
Type: text/x-diff
Size: 19569 bytes
Desc: 
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20100727/078cc453/attachment-0001.patch>