[FFmpeg-user] Towards better trims & concatenations
Mark Filipak
markfilipak.imdb at gmail.com
Sat Jan 6 21:52:24 EET 2024
Below is what I plan to submit to trac. I welcome comments and corrections.
What I wrote pretty much goes to the heart of FFmpeg, so I expect push-back.
The more people who confirm what I have written, the better.
The files are over 500 MB. If you want them, ask. But I recommend you modify the script below to
process your own 'mother'. My 'mother' was h:\BDMV\STREAM\00305.m2ts.
Towards better trims & concatenations.
-- Mark.
set MOTHER=h:\BDMV\STREAM\00305.m2ts
set SS=-ss 20.062
set TO=-to 34.076
set FILT1=setts=time_base=1/90000
set FILT2=setts=dts=DTS-STARTDTS:pts=PTS-STARTDTS
set FILT3=setts=time_base=1/90000:dts=DTS-STARTDTS:pts=PTS-STARTDTS
set VCOPY=-c:v copy
set VCODE=-vf separatefields,setdar=16/9 -c:v libx265 -x265-params crf=18
set ACOPY=-c:a copy -sn -dn
:
: The Windows settings above will make sense further down.
:
: When this command file is run, the mother: 00305.m2ts, is used to
: create the mother's clone: 00.mp4, plus 12 sons: 01.mp4..12.mp4.
: The clone and sons are created at c:\.
:
: THE FILES HOW MADE THE SIZES
: 00305.m2ts (mother) 679600 packets 15091 MB
: 00.mp4 (clone) copy (reference) 4136 packets 136 MB
: 01.mp4 (son) '-ss -i' & FILT1 1711 packets 56 MB
: 02.mp4 (son) '-ss -i' & FILT2 1711 packets 56 MB
: 03.mp4 (son) '-ss -i' & FILT3 1711 packets 56 MB
: 04.mp4 (son) '-i -ss' & FILT1 1629 packets 52 MB
: 05.mp4 (son) '-i -ss' & FILT2 1629 packets 52 MB
: 06.mp4 (son) '-i -ss' & FILT3 1629 packets 52 MB
:
: The clone, 00.mp4, is made to enable packet tracing via packet CRCs.
:
: CORRELATING THE FIRST, NON-BLACK FRAMES:
: MOTHER'S FRAME CLONE'S FRAME NOW KNOWN TO BE TRUE
: 00305.m2ts, N=36 == 00.mp4, N=36 by looking & counting frames.
:
: CORRELATING THE FIRST, NON-BLACK FRAMES:
: CLONE'S FRAME SON'S FRAME NOW KNOWN TO BE TRUE
: 00.mp4, N=508 == 01.mp4, N=27 by looking & counting frames.
: 00.mp4, N=508 == 02.mp4, N=27 by looking & counting frames.
: 00.mp4, N=508 == 03.mp4, N=27 by looking & counting frames.
: 00.mp4, N=508 == 04.mp4, N=2 by looking & counting frames.
: 00.mp4, N=508 == 05.mp4, N=2 by looking & counting frames.
: 00.mp4, N=508 == 06.mp4, N=2 by looking & counting frames.
:
: COMPUTING START FRAMES:
: CLONE'S FRAME SON'S FRAME RATIONALE
: 00.mp4, N=481 == 01.mp4, N=0 481 = 508 - 27.
: 00.mp4, N=481 == 02.mp4, N=0 481 = 508 - 27.
: 00.mp4, N=481 == 03.mp4, N=0 481 = 508 - 27.
: 00.mp4, N=504 == 04.mp4, N=0 504 = 508 - 2.
: 00.mp4, N=504 == 05.mp4, N=0 504 = 508 - 2.
: 00.mp4, N=504 == 06.mp4, N=0 504 = 508 - 2.
:
: The ffmpeg commands that made the clone and sons are in sections
: that follow.
:
: OTHER FACTS:
: There are 90000 ticks/sec FROM time_base.
: There are 1.001/24 secs/frame == 0.041708[3..] secs/frame.
: There are 90090/24 ticks/frame == 3753.75 ticks/frame.
:
: EVERYTHING ELSE IS DERIVED VIA FFMPEG TOOLS.
: That's tricky because at this point, FFmpeg tools are untrusted.
:
: TRIMMING ACCURACY.
: Timming accuracy is crucial for follow-on concatenations.
: Using MPV, a 0:20.062 trim time is found. The starting PTS is
: therefore approximately 1805580 ticks. But what is it, exactly?
: That depends on how MPV handles numbers.
: If MPV truncates, the actual trim time and PTS are in the range
: 20.062[0..]..20.062[9..] 1805580.[0..]..1805669.[9..].
: If MPV rounds, the actual trim time and PTS are in the range
: 20.0615[0..]..20.0624[9..] 1805535.[0..]..1805624.[9..].
: The only clone PTS in a range above is 1805553 -- so MPV rounds.
: The trim point is therefore 1805553 ticks, exactly.
:
: SUGGESTIONS:
: To the best of my knowledge, there's a small private data area in
: every packet. Use it to record the source's packet number into the
: target's packet. That would make packet tracing a lot surer.
: Change '-showinfo' to report real PTSs, not ticks from a starting
: PTS that often doesn't actually exist. Bogus 'PTS's are confusing.
: Add frame numbers to '-framecrc'.
: Change CRCs to cover only payloads, not whole packets.
: Find a way to differentiate silent audio packets, one from another.
: (Note that the first suggestion would accomplish it.)
: Change the default audio packet time_base to be the video packet
: time_base. Better yet, always use the same time_base. That's usual.
: If the trim frame is preceded by audio, then automatically include
: the preceding audio packet; the preceding one, only, not more.
: BEGIN
ffmpeg -to 35.118 -i %MOTHER% -map 0 -c copy -sn -dn 00.mp4
: Creating the mother's clone. The '-framecrc' below is the clone.
:
: ======================== PACKETS 0..2352 =========================
: -3754, 0, 3753, 640654, 0x07c81f9c N=0
: 0, 7507, 3753, 640652, 0xfd25cfb3 N=2
: 4 audio packets
: 3753, 3753, 3753, 640652, 0xe3dec43b N=1
: skip 2346 packets (477 video, 1869 audio)
:
: ======================= PACKETS 2353..2470 =======================
: 1798046, 1805553, 3753, 640654, 0xb9811068 N=481 - I
: 4 audio packets
: 1801800, 1801800, 3753, 640652, 0xbf9ad2d1 N=480 - B
: 4 audio packets
: 1805553, 1813061, 3753, 640652, 0x8f98895e N=483 - P
: 4 audio packets
: 1809307, 1809307, 3753, 640652, 0x37eabd4d N=482 - B
: 4 audio packets
: 1813061, 1820568, 3753, 640652, 0x343fcc78 N=485 - P
: 4 audio packets
: 1816815, 1816815, 3753, 547001, 0x82084c9d N=484 - B
: skip 92 packets (18 video & 74 audio)
:
: ========================= PACKETS 2471.. =========================
: 1888136, 1899397, 3753, 165207, 0xf6eff731 N=506 - P
: 4 audio packets
: 1891890, 1891890, 3753, 98707, 0xab3c7d53 N=504 - I
: 4 audio packets
: 1895643, 1895643, 3753, 126456, 0x7ba20402 N=505 - B
: 4 audio packets
: 1899397, 1906905, 3753, 97517, 0x44a0ecca N=508 - P
: 4 audio packets
: 1903151, 1903151, 3753, 124870, 0x9ad00c5a N=507 - B
: 4 audio packets
: 1906905, 1914412, 3753, 150721, 0x9e12de9c N=510 - P
: 4 audio packets
: 1910658, 1910658, 3753, 118479, 0x32746f50 N=509 - B
: Now it's simply a matter of making the sons and correlating the CRCs.
ffmpeg %SS% %TO% -i %MOTHER% -map 0 -bsf:v %FILT1% -bsf:a %FILT1% %VCOPY% %ACOPY% 01.mp4
: MPV says 0:00.084..0:14.635 @ 23.976fps
: +------------------+
: -29600, -29600, ¦ 47 audio packets ¦
: +-----[error1]-----+ IN 00.mp4 HERE
: -11261, -11261, 3754, 640654, 0xb9811068 N=481 N=0 - I
: 3 audio packets +------------------+
: -7507, -7507, 3753, 640652, 0xbf9ad2d1 ¦ N=480 N=1 - B ¦
: 4 audio packets +-----[error2]-----+
: -3754, -3754, 3754, 640652, 0x8f98895e N=483 N=2 - P
: 4 audio packets
: 0, 0, 3754, 640652, 0x37eabd4d N=482 N=3 - B
: 4 audio packets
: 3754, 3754, 3754, 640652, 0x343fcc78 N=485 N=4 - P
: 4 audio packets
: 7508, 7508, 3753, 547001, 0x82084c9d N=484 N=5 - B
: +-------------------------+
: frames ¦ 481 482 483 484 485 ¦
: play ¦ 481 480 483 482 485 484 ¦
: ¦ I B P B P B ¦
: +--------[error3]---------+ +-------------------------+
: '-show_streams': Video time_base=1/90000, ¦ Audio time_base=1/48000 ¦
: +--------[error4]---------+
: [error1] 47 audio packets 'appeared' that shouldn't be there.
: [error2] Frame N=480 should have been trimmed off.
: [error3] The frames are in the wrong order -- PTS errors.
: [error4] The audio time_base should be 1/90000.
ffmpeg %SS% %TO% -i %MOTHER% -map 0 -bsf:v %FILT2% -bsf:a %FILT2% %VCOPY% %ACOPY% 02.mp4
: MPV says 0:00.083..0:14.635 @ 23.976fps
: IN 00.mp4 HERE
: -37, 7470, 3753, 640654, 0xb9811068 N=481 N=1 - I
: 4 audio packets +------------------+
: 3717, 3717, 3753, 640652, 0xbf9ad2d1 ¦ N=480 N=0 - B ¦
: 4 audio packets +-----[error1]-----+
: 7470, 14978, 3753, 640652, 0x8f98895e N=483 N=3 - P
: 4 audio packets
: 11224, 11224, 3753, 640652, 0x37eabd4d N=482 N=2 - B
: 4 audio packets
: 14978, 22485, 3753, 640652, 0x343fcc78 N=485 N=5 - P
: 4 audio packets
: 18732, 18732, 3753, 547001, 0x82084c9d N=484 N=4 - B
: +-------------------------+
: '-show_streams': Video time_base=1/90000, ¦ Audio time_base=1/48000 ¦
: +--------[error2]---------+
: [error1] Frame N=480 should have been trimmed off.
: [error2] The audio time_base should be 1/90000.
ffmpeg %SS% %TO% -i %MOTHER% -map 0 -bsf:v %FILT3% -bsf:a %FILT3% %VCOPY% %ACOPY% 03.mp4
: MPV says 0:00.083..0:14.635 @ 23.976fps
: IN 00.mp4 HERE
: -37, 7470, 3753, 640654, 0xb9811068 N=481 N=1 - I
: 4 audio packets +------------------+
: 3717, 3717, 3753, 640652, 0xbf9ad2d1 ¦ N=480 N=0 - B ¦
: 4 audio packets +-----[error1]-----+
: 7470, 14978, 3753, 640652, 0x8f98895e N=483 N=3 - P
: 4 audio packets
: 11224, 11224, 3753, 640652, 0x37eabd4d N=482 N=2 - B
: 4 audio packets
: 14978, 22485, 3753, 640652, 0x343fcc78 N=485 N=5 - P
: 4 audio packets
: 18732, 18732, 3753, 547001, 0x82084c9d N=484 N=4 - B
: +-------------------------+
: '-show_streams': Video time_base=1/90000, ¦ Audio time_base=1/48000 ¦
: +--------[error2]---------+
: [error1] Frame N=480 should have been trimmed off.
: [error2] The audio time_base should be 1/90000.
: ===== Repeat 01.mp4..03.mp4, except move %SS% to after '-i' =====
ffmpeg %TO% -i %MOTHER% -map 0 %SS% -bsf:v %FILT1% -bsf:a %FILT1% %VCOPY% %ACOPY% 04.mp4
: MPV says 0:00.917..0:14.016 @ 23.976fps +---------------+
: +------------------+ ¦ N=481..N=503 ¦
: 96, 96, ¦ 86 audio packets ¦ +---[error1]----+
: +-----[error2]-----+ IN 00.mp4 HERE
: 82530, 82530, 3754, 165207, 0xf6eff731 N=506 N=0 - P
: 4 audio packets
: 86284, 86284, 3753, 98707, 0xab3c7d53 N=504 N=1 - I
: 4 audio packets
: 90037, 90037, 3754, 126456, 0x7ba20402 N=505 N=2 - B
: 4 audio packets
: 93791, 93791, 3754, 97517, 0x44a0ecca N=508 N=3 - P
: 4 audio packets
: 97545, 97545, 3754, 124870, 0x9ad00c5a N=507 N=4 - B
: 4 audio packets
: 101299, 101299, 3753, 150721, 0x9e12de9c N=510 N=5 - P
: 4 audio packets
: 105052, 105052, 3754, 118479, 0x32746f50 N=509 N=6 - B
: +-----------------------------+
: frames ¦ 504 505 506 507 508 509 510 ¦
: play ¦ 506 504 505 508 507 510 509 ¦
: ¦ P I B P B P B ¦
: +----------[error3]-----------+ +-------------------------+
: '-show_streams': Video time_base=1/90000, ¦ Audio time_base=1/48000 ¦
: +--------[error4]---------+
: [error1] The correct trim frame, N=481, was missed by 23 frames.
: [error2] 86 audio packets 'appeared' that shouldn't be there [note].
: [error3] The frames are in the wrong order -- PTS errors [note].
: [error4] The audio time_base should be 1/90000.
: [note] Does [error1] explain away [error2] & [error3]? I think not.
ffmpeg %TO% -i %MOTHER% -map 0 %SS% -bsf:v %FILT2% -bsf:a %FILT2% %VCOPY% %ACOPY% 05.mp4
: MPV says 0:00.125..0:14.016 @ 23.976fps +---------------+
: ¦ N=481..N=503 ¦
: +---[error1]----+
: IN 00.mp4 HERE
: 1888136, 1899397, 3753, 165207, 0xf6eff731 N=506 N=2 - P
: 4 audio packets
: 1891890, 1891890, 3753, 98707, 0xab3c7d53 N=504 N=0 - I
: 4 audio packets
: 1895643, 1895643, 3753, 126456, 0x7ba20402 N=505 N=1 - B
: 4 audio packets
: 1899397, 1906905, 3753, 97517, 0x44a0ecca N=508 N=4 - P
: 4 audio packets
: 1903151, 1903151, 3753, 124870, 0x9ad00c5a N=507 N=3 - B
: 4 audio packets
: 1906905, 1914412, 3753, 150721, 0x9e12de9c N=510 N=6 - P
: 4 audio packets
: 1910658, 1910658, 3753, 118479, 0x32746f50 N=509 N=5 - B
: +-------------------------+
: '-show_streams': Video time_base=1/90000, ¦ Audio time_base=1/48000 ¦
: +--------[error2]---------+
: [error1] The correct trim frame, N=481, was missed by 23 frames.
: [error2] The audio time_base should be 1/90000.
ffmpeg %TO% -i %MOTHER% -map 0 %SS% -bsf:v %FILT3% -bsf:a %FILT3% %VCOPY% %ACOPY% 06.mp4
: MPV says 0:00.125..0:14.016 @ 23.976fps +---------------+
: ¦ N=481..N=503 ¦
: +---[error1]----+
: IN 00.mp4 HERE
: -11, 11250, 3753, 165207, 0xf6eff731 N=506 N=2 - P
: 4 audio packets
: 3743, 3743, 3753, 98707, 0xab3c7d53 N=504 N=0 - I
: 4 audio packets
: 7496, 7496, 3753, 126456, 0x7ba20402 N=505 N=1 - B
: 4 audio packets
: 11250, 18758, 3753, 97517, 0x44a0ecca N=508 N=4 - P
: 4 audio packets
: 15004, 15004, 3753, 124870, 0x9ad00c5a N=507 N=3 - B
: 4 audio packets
: 18758, 26265, 3753, 150721, 0x9e12de9c N=510 N=6 - P
: 4 audio packets
: 22511, 22511, 3753, 118479, 0x32746f50 N=509 N=5 - B
: +-------------------------+
: '-show_streams': Video time_base=1/90000, ¦ Audio time_base=1/48000 ¦
: +--------[error2]---------+
: [error1] The correct trim frame, N=481, was missed by 23 frames.
: [error2] The audio time_base should be 1/90000.
More information about the ffmpeg-user
mailing list