[Ffmpeg-devel] av_seek_frame() issues with audio from MPEG-TS

[Andy Brown]

I've searched through many, many threads on av_seek_frame() issues but none
of the solutions seem to help me solve my current problem.  I'm working off
of a captured MPEG-TS file.  I begin by finding the first video key frame
with a PTS that is not AV_NOPTS_VALUE.  This will be my starting point of
playback for my new output stream.  Now I need the audio to align up exactly
so that it starts precisely when the video starts.  My idea was to take the
PTS of the video (no decoding yet so I'm still working with the AVPacket
container) and make a call into av_seek_frame() to grab the audio frame that
either starts exactly when the video frame does or just before it.

What I find is that the av_seek_frame() returns an audio frame with a PTS
greater than that of the video (by many audio frames in duration).  I
manually tweaked the PTS I provided av_seek_frame() until I found the
correct one.  In doing so I noticed that the frames it returned jumped in
PTS by value exactly equal to 12 audio frames.  12 happens to be my video
GOP size but I can't see why this would have anything to do with it.

I then called av_seek_frame() in a loop in which every iteration I provided
av_seek_frame() a slightly lower PTS (starting the first iteration with my
video PTS) hoping that eventually it would return an audio frame whose PTS
was less than or equal to my video PTS.  What I found was that on each
iteration, even though my PTS input to av_seek_frame() was decreasing, my
returned audio frame PTS was increasing.  It's almost as if it doesn't start
at the beginning of the file but instead at it's current offset.

I stepped through av_seek_frame_binary() and it looks to me like it is
suppose to start at the beginning each time.  I'm using Visual C++ 2005 so I
can't step through that portion of the code.  If no one has any ideas, I
will try to compile and run through GCC tomorrow to see if I can gather any
more information.  One thing to note, I handle the audio and video in
separate parts of my application so even though they come from the same
input file I open it twice with two separate AVFormatContext structures.  I
tried opening only one and saw similar results so I assumed that was not the
issue here.

I have a small code snippet below that shows what I trying to do.  The other
important details:
 - Visual C++ 2005
 - Code checked out from CVS on 11/29/2005
 - Applied one patch from Alexander Strasser to fix the static linking in
shared libraries
 - Input file is TS stream with MPEG-2 Video and MPEG-1 Audio

Code Snippet:

    /*Find the first Video Key Frame with a PTS*/
    for(;;) {
        av_read_frame(pVideoFC, &videoPacket);
        if(videoPacket.stream_index==videoStream) {
             if ((videoPacket.pts != AV_NOPTS_VALUE) && (videoPacket.flags &
PKT_FLAG_KEY))
                break;
        }
        av_free_packet(&videoPacket);
    }

    /* Seek to the first audio frame at or before the video pts */
    av_seek_frame(pAudioFC, audioStream, videoPacket.pts,
AVSEEK_FLAG_BACKWARD | AVSEEK_FLAG_ANY);
    for(;;) {
        av_read_frame(pAudioFC, &audioPacket);
        if(audioPacket.stream_index==audioStream) {
            break;
        }
        av_free_packet(&audioPacket);
    }

Andy