[FFmpeg-devel] Realmedia patch

[Ronald S. Bultje]

Hi Luca,

On Thu, Aug 21, 2008 at 2:56 AM, Luca Abeni <lucabe72 at email.it> wrote:
> I am not the RTSP demuxer maintainer (and I do not feel very
> comfortable with its current implementation), but I am trying
> to have a look at these patches, to help merging them.
> (BTW, I partly missed the previous threads about realmedia...
> Can you please forward me the relevant emails, or send me
> links to the ml archives, and point me to a "more split"
> patchqueue?)

That doesn't really exist, I'm very old-fashioned and don't use
anything like git or so. I just use quilt, which is basically an
elegant diff-script. Sorry... I'm willing to split in whatever way,
just as previously.

Previous threads (google my name and rtsp):

http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2007-July/033095.html
http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2007-December/039697.html

(Some parts where already committed as smaller/separate patches.)

> Ronald S. Bultje wrote:
> [...]
>>>> +        for (i = 0; i < n_streams; i++) {
>>>> +            if (i == 0)
>>>> +                stream = orig_stream;
>>>> +            else {
>>>> +                stream = av_new_stream (s, 0);
>>>> +                stream->codec->codec_type = orig_stream->codec->codec_type;
>>> missing stream != NULL check
>>> besides, this stream creation in parse_sdp_a_line() is something that
>>> should be commented by one of the lucas. I do not feel confident enough
>>> that this is the correct approuch ...
>
> I also had some doubts about this when looking at the patch (and the need
> for changing the parse_sdp_a_line() prototype looks suspect too...).
> Can you post an example SDP for which this code is needed? And a link to
> some documentation about it?
> Also, is this code needed for processing a session-level "a=" line, or
> is it a media-level SDP line?
> In the second case, I think this is the wrong approach.

It's session-level, I never found any documentation except for the
librtsp (LGPL) implementation. Here's the SDP of my test-stream (from
debugging enabled in sdp_parse_line()), some are real-specific and
Michael earlier asked for the Real-specific lines to be in rtp_rm.c,
hence the parse_sdp_a_line() forward. The reason I changed the
prototype is because the RTSPStream only points to 1 AVStream, which
this way I have the AVFormatContext and can thus add more AVStreams
depending on the amount of stream-rules:

./ffplay rtsp://helix1.smgradio.com/encoder/VirginRadioAAC.rm -stats -ast 4
[..]
sdp: v='0'
sdp: o='- 328012 328012 IN IP4 192.168.97.57'
sdp: s='Virgin Radio LIVE from London UK in Real Audio 10 AAC'
sdp: i='Virgin Radio (C) Virgin Radio Limited 2004'
sdp: t='0 0'
sdp: a='SdpplinVersion:1610641560'
sdp: a='StreamCount:integer;1'
sdp: a='Clock Start Time:integer;24833243'
sdp: a='Content Rating:integer;0'
sdp: a='Flags:integer;9'
sdp: a='IsRealDataType:integer;1'
sdp: a='LiveStream:integer;1'
sdp: a='Author:buffer;"VmlyZ2luIFJhZGlvAA=="'
sdp: a='Copyright:buffer;"KEMpIFZpcmdpbiBSYWRpbyBMaW1pdGVkIDIwMDQA"'
sdp: a='Title:buffer;"VmlyZ2luIFJhZGlvIExJVkUgZnJvbSBMb25kb24gVUsgaW4gUmVhbCBBdWRpbyAxMCBBQUMA"'
sdp: a='Audiences:string;"16k Substream for 28k Dial-up;28k
Dial-up;56k Dial-up;150k LAN;"'
sdp: a='audioMode:string;"music"'
sdp: a='Creation Date:string;"5/19/2008 15:00:10"'
sdp: a='Generated By:string;"Helix Producer SDK 10.0 for Windows,
Build 10.0.0.195"'
sdp: a='Modification Date:string;"5/19/2008 15:00:10"'
sdp: a='videoMode:string;"normal"'
sdp: m='audio 0 RTP/AVP 101'
sdp: b='AS:134'
sdp: b='RR:4800'
sdp: b='RS:1600'
sdp: a='control:streamid=0'
sdp: a='length:npt=0'
sdp: a='rtpmap:101 x-pn-multirate-realaudio-live/1000'
sdp: a='mimetype:string;"audio/x-pn-multirate-realaudio-live"'
sdp: a='AvgBitRate:integer;128000'
sdp: a='ActualPreroll:integer;2229'
sdp: a='AvgPacketSize:integer;1500'
sdp: a='EndTime:integer;0'
sdp: a='MaxBitRate:integer;128000'
sdp: a='MaxPacketSize:integer;1500'
sdp: a='MinimumSwitchOverlap:integer;200'
sdp: a='Preroll:integer;4458'
sdp: a='SeekGreaterOnSwitch:integer;0'
sdp: a='StartTime:integer;0'
sdp: a='OpaqueData:buffer;"TUxUSQAIAAAAAAABAAEAAgACAAMAAwAEAAAAVi5yYf0ABQAALnJhNQAAABAABQAAAEYAAgAAAjQAAAAAAAHaYAAAAAAACAI0AC8AAAAAViIAAFYiAAAAEAABZ2VucmNvb2sBBwAAAAAACAEAAAECAAASAAAAVi5yYf0ABQAALnJhNQAAABAABQAAAEYAAwAAAlgAAAAAAAJdmQAAAAAACQJYADwAAAAAViIAAFYiAAAAEAABZ2VucmNvb2sBBwAAAAAACAEAAAECAAAXAAAAXi5yYf0ABQAALnJhNQAAABAABQAAAE4AFQAAAdEAAAAAAAOqtAAAAAAAEAHRAF0AAAAArEQAAKxEAAAAEAACZ2VucmNvb2sBBwAAAAAAEAEAAAMIAAAgAAAAAAACAAQAAABRLnJh/QAFAAAucmE1AAAAEAAFAAAAQQACAAABcwAAAAAADqYAAAAAAAABAXMBcwAAAACsRAAArEQAAAAQAAJ2YnJmcmFhYwEHAAAAAAADAhIQ"'
sdp: a='RMFF 1.0 Flags:buffer;"AAgAAgAAAAIAAAACAAAAAgAA"'
sdp: a='ASMRuleBook:string;"#($Bandwidth <
20671),AverageBandwidth=16192,Priority=5;#($Bandwidth <
20671),AverageBandwidth=0,Priority=5,OnDepend=\"1\",
OffDepend=\"1\";#($Bandwidth >= 20671) && ($Bandwidth <
32041),AverageBandwidth=20671,Priority=5;#($Bandwidth >= 20671) &&
($Bandwidth < 32041),AverageBandwidth=0,Priority=5,OnDepend=\"2\",
OffDepend=\"2\";#($Bandwidth >= 32041) && ($Bandwidth <
128000),AverageBandwidth=32041,Priority=5;#($Bandwidth >= 32041) &&
($Bandwidth < 128000),AverageBandwidth=0,Priority=5,OnDepend=\"4\",
OffDepend=\"4\";#($Bandwidth >=
128000),AverageBandwidth=128000,Priority=5;#($Bandwidth >=
128000),AverageBandwidth=0,Priority=5,OnDepend=\"6\",
OffDepend=\"6\";"'
sdp: a='StreamName:string;"Audio Stream"'

>> So this is indeed kind of, uhm, well, hackish maybe :]. The idea is
>> that the SDP tells us which bitrates (stream rules) are possible for
>> this single RTSP stream. Per stream, you can then subscribe to these
>> stream-rules which you can read. I'm currently selecting one (I don't
>> think many is practically possible anyway). The number of rules is
>> available only in the SDP, so I need to create the additional
>> AVStreams per extra stream-rules somewhere during/after the SDP
>> reading.
> Uhmm... Wouldn't it be possible to have only one AVStream per "m="
> line, and to add something like an "available_bitrates" array (or
> similar) to RTSPStream?
> (I believe that in theory a new AVStream should be created only when
> processing an "m=" line)

That's the previous approach, yes. Michael said he'd prefer to have 1
AVStream per rule, so that the AVStream/CodecContext is filled in and
we can decide on which stream to select from by using AVStream.discard
(see ffplay). To do that, I need one AVStream per rule, not per m
line. My previous patch used the one-stream-per-m-rule approach.
There's a few reasons why that's not good:

- if you add a avail-bitrates array, you can only select based on
bitrates, which isn't very extendible
- the SDP header contains the RM header, with which I can give codec
information per stream-rule (not just bitrate, but CodecID,
stereo/mono, etc. - it could even help if new codecs arise and we
don't support all of them, we could skip those based on this
information, while we may accidently select those if we select just
based on bitrates)
- AVStream.discard was the intended API for stream selection, so this
approach fits the intended lavf-API
- [... please fill in your reason here ...]

Of course, there's also reasons why the m-line approach is better:

- you can really just play back one at a time anyway (actually, I
haven't verified this and intend to test this by broader
subscribe-rules at some point - anyone ever tried this?)
- [... please fill in your reason here ...]

> If I understand well there are not multiple different streams (one
> per bitrate), but there is a single stream for which you can select
> the bitrate... Or did I misunderstand something?

That is correct.

And thanks for helping, any input here is good! I appreciate all the
help I can get in getting this committed to svn at some point, I think
it'd be a great feature to have in whatever way. Once it's in, I'll
try to do the same to MMS.

Ronald