[Ffmpeg-devel] How to sync audi and video

[Paul Curtis]

Paul wrote:
> i have audio frame(s) and video frame in every cycle , then i want to
> write these to "output_media.mpg" file.
>
> Final, if the thought as above cannot make synch, Can you show me any
> way to sync audio and video?

[cc'ing ffmpeg-dev, as that was the original source of the message]

To interleave between video and audio, I have done the following. Assume 
that you have an AVFormatContext for output and two AVStreams, one for 
video, and one for audio.

     for(;;) {
         /* compute current audio and video time */
         if (audio_steam)
             audio_pts = (double)audio_stream->pts.val * 
audio_stream->time_base.num / audio_stream->time_base.den;
         else
             audio_pts = 0.0;

         if (video_stream)
             video_pts = (double)video_stream->pts.val * 
video_stream->time_base.num / video_stream->time_base.den;
         else
             video_pts = 0.0;

         /* write interleaved audio and video frames */
         if (!video_stream || (video_stream && audio_stream && audio_pts 
< video_pts)) {
             avcodec_encode_audio(...);
             av_write_frame(...);
         } else {
             avcodec_encode_video(...);
             av_write_frame(....);
         }
     }

When you initialize the AVFormatContext (for output) and the AVStreams, 
the values in the PTS field will be null. This code assumes you are 
starting at zero time, and moving forward. As each frame is encoded, the 
AVStream's pts.val will be updated. This allows you to write the frames 
interleaved together in a single output file.

You will have to handle, via buffering, the capture of the raw audio 
data and raw video data so you do not lose any frames. However, on a 
suitably fast machine, the encoding should be able to encode the frames 
and write in real time.

Regards,
Paul