[FFmpeg-devel] rtp streaming x264+audio issues (and some ideas to fix them)

[Martin Storsjö]

On Mon, 8 Feb 2010, Ronald S. Bultje wrote:

> In the demuxer, relative timestamps are calculated (see
> rtp_packet_finalize() in rtpdec.c) with pts = timestamp + last_ntp -
> first_ntp, each in their own timebase. For different streams within
> the same session, the ntp might have come in at a different point in
> time (and this happens for h264 streams from e.g. Axis cameras, where
> they are up to several seconds apart), causing async (up to the same
> amount, so that can be >=2 seconds under some conditions). Setting it
> to zero in read_header() fixes this (which is my curernt quickfix).
> Any ideas on a better fix for this?

Taking the nature of RTP into consideration, with stream timestamps 
starting at an random origin, I guess this approach in general makes sense 
- it gives correct sync between streams (given that their NTP timestamps 
are sensible), but the pts values start at an undefined offset.

If one would want the pts values to start at 0, I'd say the RTP demuxer 
must expose the first_rtcp_ntp_time field to the entity coordinating the 
streams (SDP/RTSP demuxer). As soon as this field has been set for one 
stream, the same value must be copied to the other streams RTP 
demuxers, so that the pts values are calculated relative to the same NTP 
time origin used as pts = 0 for the streams.

// Martin