[FFmpeg-devel] H264 video and AVPacket::duration
Michael Doilnitsyn
mdoilnitsyn at gmail.com
Tue Dec 3 20:39:49 CET 2013
On Wed, Nov 20, 2013 at 2:35 PM, Michael Niedermayer <michaelni at gmx.at <http://ffmpeg.org/mailman/listinfo/ffmpeg-devel>>wrote:
>/ On Wed, Nov 20, 2013 at 11:02:10AM -0800, Alex Sukhanov wrote:
/>/ > On Tue, Nov 19, 2013 at 5:58 PM, Michael Niedermayer <michaelni at gmx.at <http://ffmpeg.org/mailman/listinfo/ffmpeg-devel>
/>/ >wrote:
/>/ >
/>/ > > On Tue, Nov 19, 2013 at 02:27:55PM -0800, Alex Sukhanov wrote:
/>/ > > > Hi Michael and ffmpeg-devel,
/>/ > > >
/>/ > > > I recently found that for H264 video ffmpeg demuxers and ffprobe
/>/ doesn't
/>/ > > > set AVPacket::duration field.
/>/ > > > This is typical ffprobe output for H264 video packed into FLV
/>/ container:
/>/ > > >
/>/ > > >
/>/ > >
/>/ packet|codec_type=audio|stream_index=1|pts=0|pts_time=0.000000|dts=0|dts_time=0.000000|duration=23|duration_time=0.023000|convergence_duration=N/A|convergence_duration_time=N/A|size=9|pos=103|flags=K_
/>/ > > >
/>/ > >
/>/ *packet|codec_type=video|stream_index=0|pts=0|pts_time=0.000000|dts=0|dts_time=0.000000|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=4188|pos=132|flags=K_*
/>/ > > >
/>/ > >
/>/ packet|codec_type=audio|stream_index=1|pts=23|pts_time=0.023000|dts=23|dts_time=0.023000|duration=23|duration_time=0.023000|convergence_duration=N/A|convergence_duration_time=N/A|size=9|pos=4337|flags=K_
/>/ > > >
/>/ > >
/>/ *packet|codec_type=video|stream_index=0|pts=37|pts_time=0.037000|dts=37|dts_time=0.037000|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=43|pos=4366|flags=__*
/>/ > > >
/>/ > >
/>/ packet|codec_type=audio|stream_index=1|pts=46|pts_time=0.046000|dts=46|dts_time=0.046000|duration=23|duration_time=0.023000|convergence_duration=N/A|convergence_duration_time=N/A|size=9|pos=4426|flags=K_
/>/ > > >
/>/ > >
/>/ packet|codec_type=audio|stream_index=1|pts=70|pts_time=0.070000|dts=70|dts_time=0.070000|duration=23|duration_time=0.023000|convergence_duration=N/A|convergence_duration_time=N/A|size=56|pos=4452|flags=K_
/>/ > > >
/>/ > >
/>/ *packet|codec_type=video|stream_index=0|pts=73|pts_time=0.073000|dts=73|dts_time=0.073000|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=26|pos=4528|flags=__*
/>/ > > >
/>/ > >
/>/ packet|codec_type=audio|stream_index=1|pts=93|pts_time=0.093000|dts=93|dts_time=0.093000|duration=23|duration_time=0.023000|convergence_duration=N/A|convergence_duration_time=N/A|size=607|pos=4571|flags=K_
/>/ > > >
/>/ > >
/>/ *packet|codec_type=video|stream_index=0|pts=110|pts_time=0.110000|dts=110|dts_time=0.110000|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=31|pos=5198|flags=__*
/>/ > > >
/>/ > >
/>/ packet|codec_type=audio|stream_index=1|pts=116|pts_time=0.116000|dts=116|dts_time=0.116000|duration=23|duration_time=0.023000|convergence_duration=N/A|convergence_duration_time=N/A|size=599|pos=5246|flags=K_
/>/ > > >
/>/ > >
/>/ packet|codec_type=audio|stream_index=1|pts=139|pts_time=0.139000|dts=139|dts_time=0.139000|duration=23|duration_time=0.023000|convergence_duration=N/A|convergence_duration_time=N/A|size=611|pos=5862|flags=K_
/>/ > > >
/>/ > > >
/>/ > > > I found following patch, which may be related:
/>/ > > >
/>/ > >
/>/ http://git.videolan.org/?p=ffmpeg.git;a=commit;h=497431a5b645ffc39cf3acbd333c9ff0f3031adb
/>/ > > > I understand the problem with interlaced video which you fixed in
/>/ this
/>/ > > > patch, but I wanted to ask you If it's possible to move back
/>/ calculation
/>/ > > of
/>/ > > > AVPacket::duration for codecs which support Interlace?
/>/ > > >
/>/ > > > I would like to solve my problem and rollback aforementioned patch,
/>/ but
/>/ > > > before that I wanted to check with you and probably encourage some
/>/ > > > discussion.
/>/ > > > What do you think? Should I elaborate a fix in different way?
/>/ > >
/>/ > > if you have a file with mixed interlaced and progressive packets
/>/ > > does reverting the commit produce correct durations?
/>/ > > what if a parser is initialized ?
/>/ > >
/>/ > > also can you share the/a file that is affected (its always useful
/>/ > > if everyone can look at the same data insteda of everyone using a
/>/ > > different file)
/>/ > >
/>/ > > [...]
/>/ > > --
/>/ > > Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
/>/ > >
/>/ > > it is not once nor twice but times without number that the same ideas
/>/ make
/>/ > > their appearance in the world. -- Aristotle
/>/ > >
/>/ > > _______________________________________________
/>/ > > ffmpeg-devel mailing list
/>/ > >ffmpeg-devel at ffmpeg.org <http://ffmpeg.org/mailman/listinfo/ffmpeg-devel>
/>/ > >http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
/>/ > >
/>/ > > Hi,
/>/ >
/>/ > I was told that I can share file only with Michael. File sent directly to
/>/ > him. Sorry that I can not share it with everyone.
/>/ > I had some time yesterday to study FFmpeg version which we use currently
/>/ > (It's a few years old), and I found that we modified our FFmpeg and moved
/>/ > AVPacket::duration computation back. Now we are updating FFmpeg and we
/>/ > faced with this problem again. As long as we computed AVPacket::duration
/>/ > all the time, looks like it's safe to rollback
/>/ >
/>/ http://git.videolan.org/?p=ffmpeg.git;a=commit;h=497431a5b645ffc39cf3acbd333c9ff0f3031adb
/>/
/>/ what about PAFF
/>/ PAFF means mixed fields and frames
/>/ the code sets duration to a constant, how is a constant going to
/>/ equal 2 different durations ?
/>/ also a packet can contain a frame that is supposed to be displayed
/>/ for 3 fields time
/>/ i must be missing something, as to me it looks like this works just
/>/ because such cases are rare
/>/
/>/
/>/ [...]
/>/ --
/>/ Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
/>/
/>/ When the tyrant has disposed of foreign enemies by conquest or treaty, and
/>/ there is nothing more to fear from them, then he is always stirring up
/>/ some war or other, in order that the people may require a leader. -- Plato
/>/
/>/ _______________________________________________
/>/ ffmpeg-devel mailing list
/>/ ffmpeg-devel at ffmpeg.org <http://ffmpeg.org/mailman/listinfo/ffmpeg-devel>
/>/ http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
/>/
/>/
/
/ /Thu Nov 21 01:07:36 CET 2013/ /
/*Alex Sukhanov* alx.sukhanov at gmail.com <mailto:ffmpeg-devel%40ffmpeg.org?Subject=Re%3A%20%5BFFmpeg-devel%5D%20H264%20video%20and%20AVPacket%3A%3Aduration&In-Reply-To=%3CCAHufB%2BVi61XqK%3Di_FuNNVFrQL3MZUa2opDWZW2Z7jua9aOVmWw%40mail.gmail.com%3E>/ wrote:
>/ /Michael,
>/ /I think you are right: our old FFmpeg works just because it's a rare case.
>/ /In worst scenario I can copy duration computation from old ffmpeg to new
>/ /one in our private repository, but as I said, I would be happy to elaborate
>/ /some universal solution here in public repository, which would fix issue
>/ /1756 and set duration for each frame packet at the same time.
>/ /What if FFmpeg could distinguish fields and frames and reset packet
>/ /duration only if it's a field? That would work for me, because I don't deal
>/ /with fields at all. What do you think?
>/ /I'm not familiar with Ffmpeg yet, so I also would like to ask if it's
>/ /possible and how better to do that?
Hi Michael,
Alex Sukhanov kindly asked me to help with this issue cause I have essential video experience (participating in
Vanguard H.264 codec development).
Since we are talking about ff_compute_frame_duration() function, I think it is safe to suppose we should
calculate exactly/frame/ duration (not/field/ or/slice/ or something else).
H.264 streams have optional sps.vui section and store always/field/ rate, so we could simply divide it by two to obtain/frame/ rate.
Please note that H.264 stores field rate irrelevant to content, i.e. progressive, interlaced, PAFF or MBAFF stream always stores
field rate.
Proposed fix is to eliminate workaround which resets default packet duration for H.264 streams, instead provide 2x
correction to adopt for H.264 field rate.
Here is a patch:
---
libavformat/utils.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/libavformat/utils.c b/libavformat/utils.c
index 06758bf..8dde302 100644
--- a/libavformat/utils.c
+++ b/libavformat/utils.c
@@ -807,9 +807,9 @@ void ff_compute_frame_duration(int *pnum, int *pden, AVStream *st,
*pnum *= 1 + pc->repeat_pict;
}
//If this codec can be interlaced or progressive then we need a parser to compute duration of a packet
- //Thus if we have no parser in such case leave duration undefined.
+ //Thus if we have no parser, convert fieldrate to framerate and provide frame duration by default.
if(st->codec->ticks_per_frame>1 && !pc){
- *pnum = *pden = 0;
+ *pnum *= 2;
}
}
break;
--
Here is a packet level log of FLV / H264 stream before the patch:
$ ffprobe h264_aac_15sec.flv -show_packets -print_format compact | grep video
ffprobe version 2.1.1 Copyright (c) 2007-2013 the FFmpeg developers
built on Nov 30 2013 13:53:10 with gcc 4.8.2 (GCC)
configuration:
libavutil 52. 48.101 / 52. 48.101
libavcodec 55. 39.101 / 55. 39.101
libavformat 55. 19.104 / 55. 19.104
libavdevice 55. 5.100 / 55. 5.100
libavfilter 3. 90.100 / 3. 90.100
libswscale 2. 5.101 / 2. 5.101
libswresample 0. 17.104 / 0. 17.104
Input #0, flv, from 'h264_aac_15sec.flv':
Duration: 00:00:15.53, start: 0.000000, bitrate: 346 kb/s
Stream #0:0: Video: h264 (Main), yuv420p, 854x480 [SAR 1:1 DAR 427:240], 27.25 tbr, 1k tbn, 54.46 tbc
Stream #0:1: Audio: aac, 44100 Hz, stereo, fltp
packet|codec_type=video|stream_index=0|pts=0|pts_time=0.000000|dts=0|dts_time=0.000000|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=4188|pos=132|flags=K
packet|codec_type=video|stream_index=0|pts=37|pts_time=0.037000|dts=37|dts_time=0.037000|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=43|pos=4366|flags=_
packet|codec_type=video|stream_index=0|pts=73|pts_time=0.073000|dts=73|dts_time=0.073000|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=26|pos=4528|flags=_
packet|codec_type=video|stream_index=0|pts=110|pts_time=0.110000|dts=110|dts_time=0.110000|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=31|pos=5198|flags=_
packet|codec_type=video|stream_index=0|pts=147|pts_time=0.147000|dts=147|dts_time=0.147000|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=31|pos=6493|flags=_
packet|codec_type=video|stream_index=0|pts=184|pts_time=0.184000|dts=184|dts_time=0.184000|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=24|pos=7143|flags=_
packet|codec_type=video|stream_index=0|pts=220|pts_time=0.220000|dts=220|dts_time=0.220000|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=30|pos=8355|flags=_
packet|codec_type=video|stream_index=0|pts=257|pts_time=0.257000|dts=257|dts_time=0.257000|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=34|pos=9453|flags=_
packet|codec_type=video|stream_index=0|pts=294|pts_time=0.294000|dts=294|dts_time=0.294000|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=29|pos=10023|flags=_
packet|codec_type=video|stream_index=0|pts=331|pts_time=0.331000|dts=331|dts_time=0.331000|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=27|pos=11129|flags=_
packet|codec_type=video|stream_index=0|pts=367|pts_time=0.367000|dts=367|dts_time=0.367000|duration=N/A|duration_time=N/A|convergence_duration=N/A|convergence_duration_time=N/A|size=24|pos=11687|flags=_
Here is a packet level log of FLV / H264 stream after the patch:
$ ffprobe h264_aac_15sec.flv -show_packets -print_format compact | grep video
ffprobe version 2.1.1 Copyright (c) 2007-2013 the FFmpeg developers
built on Nov 30 2013 13:53:10 with gcc 4.8.2 (GCC)
configuration:
libavutil 52. 48.101 / 52. 48.101
libavcodec 55. 39.101 / 55. 39.101
libavformat 55. 19.104 / 55. 19.104
libavdevice 55. 5.100 / 55. 5.100
libavfilter 3. 90.100 / 3. 90.100
libswscale 2. 5.101 / 2. 5.101
libswresample 0. 17.104 / 0. 17.104
Input #0, flv, from 'h264_aac_15sec.flv':
Duration: 00:00:15.53, start: 0.000000, bitrate: 346 kb/s
Stream #0:0: Video: h264 (Main), yuv420p, 854x480 [SAR 1:1 DAR 427:240], 27.75 fps, 27.25 tbr, 1k tbn, 54.46 tbc
Stream #0:1: Audio: aac, 44100 Hz, stereo, fltp
packet|codec_type=video|stream_index=0|pts=0|pts_time=0.000000|dts=0|dts_time=0.000000|duration=36|duration_time=0.036000|convergence_duration=N/A|convergence_duration_time=N/A|size=4188|pos=132|flags=K
packet|codec_type=video|stream_index=0|pts=37|pts_time=0.037000|dts=37|dts_time=0.037000|duration=36|duration_time=0.036000|convergence_duration=N/A|convergence_duration_time=N/A|size=43|pos=4366|flags=_
packet|codec_type=video|stream_index=0|pts=73|pts_time=0.073000|dts=73|dts_time=0.073000|duration=36|duration_time=0.036000|convergence_duration=N/A|convergence_duration_time=N/A|size=26|pos=4528|flags=_
packet|codec_type=video|stream_index=0|pts=110|pts_time=0.110000|dts=110|dts_time=0.110000|duration=36|duration_time=0.036000|convergence_duration=N/A|convergence_duration_time=N/A|size=31|pos=5198|flags=_
packet|codec_type=video|stream_index=0|pts=147|pts_time=0.147000|dts=147|dts_time=0.147000|duration=36|duration_time=0.036000|convergence_duration=N/A|convergence_duration_time=N/A|size=31|pos=6493|flags=_
packet|codec_type=video|stream_index=0|pts=184|pts_time=0.184000|dts=184|dts_time=0.184000|duration=36|duration_time=0.036000|convergence_duration=N/A|convergence_duration_time=N/A|size=24|pos=7143|flags=_
packet|codec_type=video|stream_index=0|pts=220|pts_time=0.220000|dts=220|dts_time=0.220000|duration=36|duration_time=0.036000|convergence_duration=N/A|convergence_duration_time=N/A|size=30|pos=8355|flags=_
packet|codec_type=video|stream_index=0|pts=257|pts_time=0.257000|dts=257|dts_time=0.257000|duration=36|duration_time=0.036000|convergence_duration=N/A|convergence_duration_time=N/A|size=34|pos=9453|flags=_
packet|codec_type=video|stream_index=0|pts=294|pts_time=0.294000|dts=294|dts_time=0.294000|duration=36|duration_time=0.036000|convergence_duration=N/A|convergence_duration_time=N/A|size=29|pos=10023|flags=_
packet|codec_type=video|stream_index=0|pts=331|pts_time=0.331000|dts=331|dts_time=0.331000|duration=36|duration_time=0.036000|convergence_duration=N/A|convergence_duration_time=N/A|size=27|pos=11129|flags=_
packet|codec_type=video|stream_index=0|pts=367|pts_time=0.367000|dts=367|dts_time=0.367000|duration=36|duration_time=0.036000|convergence_duration=N/A|convergence_duration_time=N/A|size=24|pos=11687|flags=_
Also I tested several H264 / PAFF streams - packet durations become correct after the patch.
Looking forward to receive your comments.
Thank you,
Best Regards,
Mike Doilnitsyn
More information about the ffmpeg-devel
mailing list