[FFmpeg-devel] [PATCH] Add libavformat option to ignore non monotime dts timestamps

[Sam Creasey]

On Fri, Sep 24, 2010 at 1:37 PM, Michael Niedermayer <michaelni at gmx.at> wrote:
> On Fri, Sep 24, 2010 at 01:14:09PM -0400, Sam Creasey wrote:
>> On Fri, Sep 24, 2010 at 12:10 PM, Mike Scheutzow
>> <mike.scheutzow at alcatel-lucent.com> wrote:
>> > Sam Creasey wrote:
>> >>
>> >> This patch adds an option which disables an error check for dts
>> >> ordering in compute_pkt_fields2(). ?This was needed to store the RTP
>> >> stream from a particular model camera usefully (using the existing
>> >> igndts flag resulted in rather jumpy playback of the final stream).
>> >> Should have no effect if the new option is not specified.
>> >
>> > I am wondering if this is a reasonable way to solve the problem. It seems
>> > like this feature could lead to the creation of an even more badly-formed
>> > output file.
>> >
>> > Please give us more info:
>> >
>> > 1. what is the format of the RTP payload? Is it a well-known container type?
>>
>> The RTP payload is H264 video, using payload type 96 (through rtpmap
>> in the SDP). ?AFAICT, the camera in question is using a standard H.264
>> container, as most tools/players seem to be able to record/play the
>> stream. ?I haven't done the RFC diving to actually hand-decode the
>> incoming packets.
>>
>> > 2. what container type are you writing the output to?
>>
>> ASF, though this particular patch is also needed for AVI output to
>> work. ?I haven't tried any other containers.
>>
>> > 3. is the failing timestamp equal to the preceeding one, or is the pkt->dts
>> > value actually smaller?
>>
>> Equal.
>>
>> > 4. do you know if the container already has these "bad" DTS timestamps, or
>> > are they being created by FFmpeg itself?
>>
>> Well, that led me down a much more complicated road of investigation
>> than "this error seems to be causing me problems, I'll just turn it
>> off..."
>>
>> They appear to be generated by FFmpeg itself, particularly in
>> ffmpeg.c:output_packet(), on line 1748 during the call to
>> av_rescale_q() in the case that the input packet's dts was equal to
>> AV_NOPTS_VALUE. ?In the most reproducable case, av_rescale_q returns 1
>> for two packets in a row, which eventually causes the check in
>> compute_pkt_fields2() to fail.
>>
>> I've seen other cases where the failure was of a repeated dts of other
>> larger numbers (50, in one case), though that occured before I'd
>> instrumented to code sufficiently to see where the dts was coming
>> from.
>>
>> As an additonal note, the case of the duplicated timestamp appears to
>> only occur when both the dts and pts values of the pkt argument to
>> output_frame() are AV_NOPTS_VALUE. ? In the alternate case, whatever
>> is generating the input to output_frame() begins assigning values to
>> pts, and the case I kludged around by ignorning the later error is
>> avoided.
>>
>> I have a strong suspicion that having found something slightly closer
>> to a root cause, my odds of getting a patch accepted which simply
>> papers over it are quite low, and the odds of my having to spend many
>> more hours in this code have increased dramatically...
>>
>> I'll continue digging and see if there's a more "proper" way to
>> resolve this issue.
>
> do the packets have a duration set?
> if yes why is it not used to extrapolate dts?
> if no, why not?
>
> also make sure the timestamps are not rounded down to lower precission that
> too can make 2 equal

I can give you a little more information on this.  The problem
basically seems to be around the following calculation in
output_frame():

if (pkt->dts == AV_NOPTS_VALUE)
    opkt.dts = av_rescale_q(ist->pts, AV_TIME_BASE_Q, ost->st->time_base);

When ist->pts has incremented to the point that av_rescale_q returns 1
for two consecutive frames, ffmpeg bombs out on the monotonic
timestamp check.  Often, this is avoided because dts == 0 is special
cased in the test, and the input frames will start having a valid dts
before its->pts has incremented sufficiently to cause a problem.

I'm not sure that I can put in a lot of additional time to finding a
proper fix for this case, as I'm already well over the time budget I'd
had to get ffmpeg working with the input from this camera.  If there
are some additonal changes or investigations which would be helpful, I
can let you know, but I doubt I'm going to be able to unravel the code
used to timestamp the incoming frames with just another couple hours
of digging.

-- Sam