[FFmpeg-devel] rtp streaming x264+audio issues (and some ideas to fix them)

[Luca Abeni]

Hi,

On Sat, 2010-02-06 at 12:17 +0200, Timo Ter?s wrote:
[...]
> Now the second problem was that I was getting A-V sync problems.
> It would appear that the RTP destinations do not synchronize each
> other at all. And since the x264 encoder introduces ~2s delay
> things get out of sync. (If I just recorded to a mp4 file or so
> a-v sync was maintained).
[...]
> --- libavformat/rtpenc.c	(revision 21651)
> +++ libavformat/rtpenc.c	(working copy)
> @@ -91,7 +91,8 @@
>      s->cur_timestamp = 0;
>      s->ssrc = 0; /* FIXME: was random(), what should this be? */
>      s->first_packet = 1;
> -    s->first_rtcp_ntp_time = AV_NOPTS_VALUE;
> +    //s->first_rtcp_ntp_time = AV_NOPTS_VALUE;
> +    s->first_rtcp_ntp_time = ntp_time();
>  
>      max_packet_size = url_fget_max_packet_size(s1->pb);
>      if (max_packet_size <= 12)

Thanks for investigating and reporting this issue. I'll check your patch
in next week.


> And indeed, after this vlc (with large enough file-cache) seemed
> to play the live stream pretty well. However, it'd be better
> if the RTP streams were synchronized so there isn't any manual
> need to twiddle with the player cache sizes (and QT did not
> like the stream in this case either).
> 
> What would be the proper way to fix the synchronization issue?
> So that the RTP packets sent out are approximately (<1s difference)
> from the original stream.

I think the ffmpeg program works well as a streamer in 90% of the cases,
but here we are in the remaining 10% :)
If you want to use the ffmpeg program to stream, I think you have to
configure the libx264 options to reduce the encoding latency to a
smaller value.
Otherwise, you can use a proper streamer which buffers the audio and
video packets, and sends them out at the proper times. For example, have
a look at AVStreamer: http://imedia.disi.unitn.it/software.html
you can encode the audio and video with ffmpeg, mux them in a NUT
container (or something similar), and then send it to avstreamer through
a pipe. I know some people are using a similar setup, and it works well.


				Luca