[FFmpeg-devel] [PATCH] avformat/movenc: Fix tfdt out of sync

[胡玮文]

Hi Martin,

OK, here I provide some codes to demo this bug. Many of these are copied from libavformat/tests/movenc.c

#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>
#include <libavutil/opt.h>
#include <libavutil/intreadwrite.h>

static const uint8_t h264_extradata[] = {
    0x01, 0x4d, 0x40, 0x1e, 0xff, 0xe1, 0x00, 0x02, 0x67, 0x4d, 0x01, 0x00, 0x02, 0x68, 0xef
};

AVPacket *pkt;
AVFormatContext *ctx;

void mux_packet(int64_t ts) {
    uint8_t pktdata[8] = { 0 };

    av_packet_unref(pkt);
    pkt->stream_index = 0;
    pkt->pts = pkt->dts = ts;
    AV_WB32(pktdata + 4, pkt->pts);
    pkt->data = pktdata;
    pkt->size = 8;

    av_write_frame(ctx, pkt);
}

int main() {
    avformat_alloc_output_context2(&ctx, NULL, NULL, "bug.mp4");
    // Setting this flag is important
    av_opt_set(ctx, "movflags", "+frag_every_frame", AV_OPT_SEARCH_CHILDREN);
    AVStream *st = avformat_new_stream(ctx, NULL);
    st->codecpar->codec_type = AVMEDIA_TYPE_VIDEO;
    st->codecpar->codec_id = AV_CODEC_ID_H264;
    st->codecpar->width = 640;
    st->codecpar->height = 480;
    st->time_base.num = 1;
    st->time_base.den = 30;
    st->codecpar->extradata_size = sizeof(h264_extradata);
    st->codecpar->extradata = av_mallocz(st->codecpar->extradata_size + AV_INPUT_BUFFER_PADDING_SIZE);
    memcpy(st->codecpar->extradata, h264_extradata, sizeof(h264_extradata));
    avio_open(&ctx->pb, "bug.mp4", AVIO_FLAG_WRITE);
    avformat_write_header(ctx, NULL);

    pkt = av_packet_alloc();
    mux_packet(0);
    mux_packet(1);
    mux_packet(2);
    av_write_frame(ctx, NULL); // Manually flush a frag
    mux_packet(10);            // Automatically flush an empty frag, because frag_every_frame is set
    mux_packet(11);
    mux_packet(12);

    av_write_trailer(ctx);
    avformat_free_context(ctx);
    av_packet_free(&pkt);
}

After compile and run this code, run this command to inspect the dts (which comes from the out of sync tfdt)

ffprobe -show_packets bug.mp4 | grep dts=

The output is:

dts=0
dts=1
dts=2
dts=2
dts=3
dts=4

With this patch applied, the output is:

dts=0
dts=1
dts=2
dts=10
dts=11
dts=12

Regards,
Hu Weiwen

On Mon, Jul 19, 2021 at 11:33:59PM +0300, Martin Storsjö wrote:
> Hi,
> 
> Thanks for the patch! I'll try to look into it in a while (I'm a bit swamped
> and short on time at the moment), hopefully within a couple days or so.
> 
> On Sun, 18 Jul 2021, Hu Weiwen wrote:
> 
> > Fix an edge case when auto flushing an empty fragment, and the previous fragment
> > has inaccurate duration, the track->frag_start would be out of sync from input
> > dts, and all subsequent tfdt box will have out of sync base_media_decode_time.
> > This error can accumulate to quite large over long-running streaming.
> > 
> > This can be easily reproduced by remux a variable frame rate source with dash
> > muxer and set streaming to 1.
> 
> Can you provide a specific sample and a command line, and point out what
> values in the output file that are mismatched - so I can zoom in on the
> issue quicker once I have time to sit down and look at it?
> 
> // Martin