[FFmpeg-devel] [PATCH v2] fftools/ffmpeg: fix video frame lost sync for the dual input ts files with audio

[Limin Wang]

On Fri, Jul 12, 2019 at 09:46:01PM +0200, Michael Niedermayer wrote:
> On Thu, Jul 11, 2019 at 09:54:29AM +0800, lance.lmwang at gmail.com wrote:
> > From: Limin Wang <lmwang at bravovcloud.com>
> > 
> > How to reproduce the problem(use two ts files with audio stream):
> > ffmpeg -i left_w_a.ts -i right_w_a.ts -filter_complex "hstack=inputs=2" -f null -
> > 
> > With above command, the audio stream of the second input will be discarded default, however the start_time is
> > initalized from the audio stream, so the old code will try to correct the ts offset in the condition which will
> > cause the video frame out of sync and report some dts error like below:
> > [null @ 0x7fa78c000c00] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 43 >= 43
> > [null @ 0x7fa78c000c00] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 44 >= 44
> > [null @ 0x7fa78c000c00] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 45 >= 45
> > [null @ 0x7fa78c000c00] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 46 >= 46
> > [null @ 0x7fa78c000c00] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 47 >= 47
> > 
> 
> > Or you can try with below command to reproduce the issue:
> > ffmpeg -i input_w_a.ts -i input_w_a.ts -filter_complex "ssim=f=ssim.log" -f null -
> > 
> > You'll get below ssim result which isn't expected result for the two input is same, the SSIM should be 1.00000
> > SSIM Y:0.777353 (6.523822) U:0.964087 (14.447539) V:0.966739 (14.780661) All:0.840039 (7.959872)
> > 
> > With the patch applied, the above testing command can get expected result.
> > 
> > Other solution are: use the setpts=PTS-STARTPTS filter or skip all audio stream to avoid the timestamp adjustment.
> > 
> > Have passed "make fate SAMPLES=fate-suite" with mpeg2-ticket186 ref data is updated.
> > 
> 
> > Reviewed-by: Michael Niedermayer <michael at niedermayer.cc>
> 
> I have not really reviewed this 
> 
> this:
> ./ffmpeg -i fate-suite/h264/h264_intra_first-small.ts  -bitexact -an -t 1 -f framecrc -
> 
> produces empty output after the patch but before it produces 14 frames
> 
For the old code, the video start time will be adjusted 1356500, it's
about 1.35s, that's means the audio is delay about -1.35s relative to
video. However now we don't use audio stream, so the patch consider 
it's better to keep video start time. To get the same result as before, 
we had adjust the -t 1 to -t 2.35 like below command:

./ffmpeg -i fate-suite/h264/h264_intra_first-small.ts  -bitexact -an -t
2.35 -f framecrc - 


> Thanks
> 
> [...]
> -- 
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
> 
> Those who are best at talking, realize last or never when they are wrong.



> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".