[FFmpeg-devel] Audio Video Sync issue with dvr-ms files

[Jonathon Elwood]

On 5/12/07, Michael Niedermayer <michaelni at gmx.at> wrote:
> Hi
>
> On Sat, May 12, 2007 at 09:15:33AM -0400, Jonathon Elwood wrote:
> > On 5/12/07, Michael Niedermayer <michaelni at gmx.at> wrote:
> > >Hi
> > >
> > >On Fri, May 11, 2007 at 11:26:00PM -0400, Jonathon Elwood wrote:
> > >> On 5/11/07, Michael Niedermayer <michaelni at gmx.at> wrote:
> > >> > Hi
> > >>  [...]
> > >>
> > >> > ive tried these files and i cannot reproduce any freezing frames with
> > >> > svn head ffplay with them, also their AV sync is not worse with ffplay
> > >> > then any other random mpeg2 video
> > >> >
> > >> > can someone confirm that there is any problem at all with these files?
> > >> You are correct, I just tried it with ffplay and it played perfectly.
> > >> I'm embarrassed that I didn't try ffplay earlier (I thought of trying
> > >> it with MPlayer, but not ffplay).  Anyway, I think this helps narrow
> > >> down the problem because the problem must be somewhere in the encoding
> > >> process.  Please try it again with ffmpeg and you should see the
> > >> problem.  I tried it with both command lines below and I see the
> > >> problem with both encoders.
> > >>
> > >> ffmpeg -benchmark -y -i "ABC_SD.dvr-ms" -vcodec mpeg4 -qscale 4 -vtag
> > >> XVID -acodec mp3 -ac 2 -ab 128000 -f avi "ABC_SD.avi"
> > >>
> > >> ffmpeg -y -benchmark -i "ABC_SD.dvr-ms" -threads 2 -vcodec h264 -b
> > >> 384000 -bufsize 384 -coder 0 -level 13 -acodec aac "ABC_SD.mp4"
> > >
> > >-dts_delta_threshold 1
> > >
> > >though this is more a workaround than a proper solution, the problem is
> > >likely that the first frames have no timestamps
> > >maybe theres some initial timestamp hidden somewhere in asf?
> > >if not iam wondering how wmp can play it? or does wmp discard the first
> > >frame(s) ?
> > >
> > The ?dts_delta_threshold 1 option kind of works?  The video isn't
> > frozen for 8 seconds, but the AV isn't in sync.
> > I tried playing the samples in wmp and I don't see wmp discarding any
> > frames.  Actually, when I play the file in wmp or ffplay it looks
> > exactly the same.  Since they look the same, it doesn't look like
> > either wmp or ffplay throws away the first frames (unless they both
> > are throwing away the same frames...).
> > I've attached two debug files that are interesting.  The first, is
> > ABC_HD.txt.  This shows the timestamps from a show that ffmpeg can
> > convert just fine.  Notice the difference from the first audio
> > timestamp to the first video timestamp is 660.  Then the second file
> > is the ABC_SD.txt file.  This is the debug from the ABC_SD.dvr-ms
> > file.  The difference here between the first audio timestamp and the
> > first video timestamp is 707.  Not a big difference?  I wouldn't
> > expect this difference to cause the first frame to be stuck for 8
> > seconds on the ABC_SD file, but everything to convert just fine with
> > the ABC_HD file (if you want to see the ABC_HD file let me know and
> > I'll upload it).
> > Anyway, I can keep looking, but I don't think there are any other
> > timestamps in the file.  Since wmp and ffplay can both play the file
> > just fine without the beginning timestamps, my guess is no other
> > timestamps exist because they aren't needed just to play the file.
> > However, since they appear to be needed for encoding, what would be
> > the proper thing to do with the timestamps?  I tried setting the first
> > video packet to a timestamp of zero, but that didn't work.  All the
> > packet fragments we get before the first timestamp is all one segment.
> > Because of this, I don't think there is any way to interpolate the
> > timestamps before we get to the real timestamps.
>
> there is you just have to subtract the timestamp discontinuity at the
> occurance of the first valid timestamp from the initial cur_dts
> somewhere in av_find_stream_info()
> also see the disscussions with wolfram on the mailinglist, some
> of them where related to this, though not related to dvr-ms
 [...]

I've made a little progress in figuring out how to set the initial
Video timestamp in DVR-MS files.  However, there are a few I can't
figure out.  Currently, the audio detection logic resets the streams,
however, that reset doesn't seem to be working for me.  Before the
reset I'm setting the first timestamp to UNKNOWN (because it is a -1),
and after the reset I calculate the correct value for this timestamp
and set it, however, the parser reports it as UNKNOWN still.  I'm
assuming this is because the timestamps were buffered somewhere on the
first pass.
My questions are?
Should I just use the "audio detection" logic's stream reset, or
should I perform another reset?  If I just rely on the "audio
detection" for the reset, then how do I flush the buffer that is
holding the timestamps?  If I don't rely on the audio detection for
the reset, then how do I perform a reset?