[FFmpeg-user] Video jumps after segmenting, transcoding, and rejoining video

[Ted Park]

> We're working on a project to have a highly parallelized video rescaling
> pipeline but we're finding that small audio and video jumps are being
> introduced into the video when we put the video back together. Can anyone
> point out anything we're doing wrong?

The video scaling is the main reason to parallelize (distribute?) the transcoding process right? Try only splitting/transcoding/concatenating the video stream in segments, then transcoding the audio separately. I don’t think it’ll be necessary to do this in segments so I’d recommend just transcoding the whole audio stream in one go but segmenting it shouldn’t be a problem if you really want. (Probably)

> The pipeline has 3 stages. First we split the video using the segment muxer
> $ ./ffmpeg -i samples/2160p-h264-aac-20mbs-2m.mp4 -muxdelay 0 -muxpreload 0
> -g 96 -keyint_min 96 -sc_threshold 0 -c copy -copyts -f segment -fflags
> +genpts -segment_time 4 -segment_format mpegts out/segments/%04d.ts

You’re codec copying so most of these options don’t really make sense I believe. And if you separate the video stream, it obviates the muxdelay and muxpreload options I think. Try:

$ ./ffmpeg -i samples/2160p-h264-aac-20mbs-2m.mp4 -an -c copy -f segment -segment_list segment_list.txt -segment_list_format ffconcat -segment_time 4 -segment_time_delta 1 -segment_format mpegts out/segments/%04d.ts

The -segment_time_delta should allow some variation in segment lengths so it doesn’t cut at exactly 4 seconds. If you’re transcoding individual segments that doesn’t sound like it would work too well.

The segment list allows for easier concatenation later.

I don’t know the reasoning behind copying timestamps, but if it’s needed for some reason, then copy the ts timestamps by adding -segment_format_options mpegts_copyts=1

> Then we change the scale for each of the segments using:
> $ ./ffmpeg -i out/segments/0000.ts -muxdelay 0 -vf scale=1920:1080 -c:v
> libx264 -profile:v main -c:a aac -copyts out/segments/out00.ts

Transcode/scale the video only segments like so

$ ./ffmpeg -i out/segments/0000.ts -vf scale=1920:1080 -c:v
libx264 -profile:v main out/segments/out00.ts

> And then we rejoin
> $ ./ffmpeg -i
> "concat:out/segments/out00.ts|out/segments/out01.ts|out/segments/out02.ts"
> -c copy -copyts out/transcoded.mp4
> 
And when you concatenate, use the concat demuxer instead of the protocol.

$ ffmpeg -f concat -i segment_list.txt -i samples/2160-h264-aac-20mbs-2m.mp4 -map 0:v -map 1:a -c:v copy -c:a ac3 output3.mp4

Obviously add whatever audio encoding parameters you want at this point

> For reference the commands are using ffmpeg 4.1.3 and the video I tested
> the commands with
> http://downloads.4ksamples.com/videos/PUPPIES%20BATH%20IN%204K%20(ULTRA%20HD)(Original_H.264-AAC)%20(4ksamples.com).mp4

Also I’m not entirely sure about this, but the framerate on this video seems very strange. Maybe you should check that out also.