[FFmpeg-devel] [RFC] Synchronize the start times of multiple RTP streams

[Martin Storsjö]

On Fri, 2 Apr 2010, Ronald S. Bultje wrote:

> I'm starting to believe I was wrong here, also because my earlier
> patch broken RTSP seeking. utils.c works based on
> AVStream->start_time, rebasing it to a global timebase, and then
> setting AVFormatContext->start_time to that. We set (I think validly)
> these AVStream->start_time values to the range: value in the SDP,
> which has no relation to RTP header timestamps. :-(. That's why it all
> doesn't work the way it should.
> 
> I think we should revive your patch, with some fixes:
> - we should test/verify it fixes seeking (in ffplay)
> - I'm not quite sure why you do this under if (ret < 0), I think that
> if can be removed, and in fact I'm wondering if you should only use it
> if ret == 0/1 (?)

The reason for doing this only in ret < 0 is that receiving a RTCP packet 
makes the parse routine return < 0 - and only if we received a RTCP 
packet, the timestamps may have been initialized, so it isn't necessary to 
check for the case when the first sender report was received otherwise.

> - for many streams, the first RTCP packet only comes after a bunch of
> RTP packets. We want to make sure that after parsing, when they can be
> assigned a timestamp, this doesn't get < 0

I'm not sure I understand what you're referring to at the end of this 
sentence. The packets that have been received before the first RTCP have 
been passed out from the RTSP code already and can't be updated anymore, 
right?

As Luca pointed out, RTSP also passes a response header RTP-Info, in 
response to PLAY, that actually gives the start RTP timestamp for all 
streams. (This only is given for non-live, seekable streams, though.)

So one way to get proper timestamps for all packets from the beginning, 
would be to use the RTP-Info header, if available. According to the RFC, 
one should still use the later RTCPs for resyncing and taking potential 
drift into account.

So, the complete solution perhaps would be to initially parse RTP-Info, if 
available, use that as initial sync point for timestamps in rtpdec - then 
when the first RTCP is received, sync that to the previously used sync 
origin, and after that only use RTCP. I can try to cook up some patch that 
does this. But except for using the RTP-Info if available, I think the 
basic solution boils down to more or less my previous patch.

As for seeking actually working - with the patch I sent earlier, it 
somewhat seems to work. Seeking forwards seems to work quite ok, but 
seeking backwards doesn't really work that well at all.

I'll try to look more into it...

// Martin