[FFmpeg-user] optimal exact segmentation without re-encoding

Thu Jul 6 23:01:05 EEST 2023

El jue, 6 jul 2023 a las 12:16, David Bernat (<david.bernat at gmail.com>)
escribió:

> At the risk of being rude, Eduardo Alarcón, (and me top-commenting), please
> do not thread-hijack.
>
How is this thread-hijack? do you remember this is a mailing list, right? i
have done something similar to what you want to do, so i'm telling you what
i did to help.

Carl Zwanzig and I are having a very careful conversation and I wish to
> continue with him.
> The use of the SEGMENT command requires re-encoding, in my experience, even
> though each segment can be set to go from keyframe to keyframe.
> If this is incorrect, I look to Carl Zwanzig for more information at this
> time, commiserate with my detailed reply. Thank you both.
>

As per example on the documentation, it doesn't require re-encoding :

Segment the input file, and create an M3U8 live playlist (can be used as
live HLS source):
ffmpeg -re -i in.mkv -codec copy -map 0 -f segment -segment_list
playlist.m3u8 -segment_list_flags +live -segment_time 10 out%03d.mkv

You can use -codec copy and do not need to reencode, ffmpeg will try find
the closet keyframe to the segment time you defined without re-encoding, as
you asked.

DB
>
>
> > On Wed, Jul 5, 2023 at 9:39 PM Carl Zwanzig <cpz at tuunq.com> wrote:
> > >
> > > > On 7/5/2023 3:29 PM, David Bernat wrote:
> > > > > Premise: segment a mov file into about one second segments without
> > > > > re-encoding; yet preserving concatenation; such that the
> segmentation
> > > is
> > > > > embarrassingly parallel, for high-speed segmenting.
> > > >
> > > > What is the purpose of these segmented files (how will they be used)?
> > Are
> > > > they going to be processed and then assembled together (which may
> > involve
> > > > decode/recode operations). Depending on the unknown intermediate
> work,
> > it
> > > > may be faster overall to decode all into an uncompressed state and
> work
> > > on
> > > > that before concatenating into a reencode pass (which could also be
> > done
> > > > in
> > > > segments which are then themselves concatenated).
> > > >
> > >
> > > The parallelization of processing is an essential motivation. (Thank
> you
> > > and looking forward to forwarding this discussion and solution with
> you.)
> > >
> > > Multiple uses but may we focus on two:
> > >
> > > 1. computer vision and thumbnails:
> > >
> > > Object detection runs quickly on images; and, in most cases, it is
> > > sufficient to summarize a long video by applying object detection
> every 1
> > > or 2 seconds.
> > > Extracting a frame using -ss is very fast but not immediate: my speed
> > tests
> > > indicate that a CPU can achieve about 60 image extractions in about 15
> > > seconds.
> > > ffmpeg -ss can be embarrassingly parallelized and does speed up from
> > about
> > > 18 seconds to 15 seconds.
> > > The precise timestamp of the image is not essential (within a few
> frames
> > is
> > > more than sufficient) and so ffmpeg -ss is sufficient.
> > > Accelerating this even further is a wonderful benefit.
> > > Furthermore, if the images are already segmented, each segment can be
> > > handled separately on a different cloud unit or CPU.
> > > In this case, motivating a storage option that is segmented is the key.
> > > Notice that this method does not strictly require that recomposition is
> > > achievable.
> > >
> > > 2. storage and generic processing:
> > >
> > > This adds the additional requirement that recomposition is achievable.
> > > Numerous use cases apply the parallelization scheme described above.
> > > It would be hugely beneficial for the storage efficiencies gained from
> > the
> > > above also constitute the full storage of the video.
> > > In an extreme case, you can imagine a video player, in which the one
> > second
> > > video clips are also serving a make-shift streaming solution.
> > > Whether this is the best example of that use case the concept is
> > sufficient
> > > and additional use cases are welcomed.
> > >
> > >
> > > > Also, please post some of the commands you've used.
> > > >
> > >
> > > Here is an overview of the processes I am using at the moment:
> > >
> > > *This series of commands identifies the KEYFRAMES.*
> > >
> > > # cmd = "ffprobe -loglevel error -select_streams v:0 -show_entries
> > > packet=pts_time,flags -of csv=print_section=0 IMG_9209.MOV | awk -F','
> > > '/K/ {print $1}'"
> > > # result = subprocess.run(cmd, shell=True, cwd=os.getcwd(),
> > > capture_output=True)
> > >
> > > # key_frames = [float(t) for t in result.stdout.decode().split()] ==>
> > > [0.000000, 1.068333, 2.135000, 3.211667, 4.295000]
> > >
> > > This command creates SEGMENTs from KEYFRAME to KEYFRAME but is not
> > > precisely exact, and hence fails the concatenation requirement.
> > >
> > > # cmd = "ffmpeg -y -ss {key_frames[i]} -to {key_frames[i+1]} -i
> > > IMG_9209.MOV -c copy segment_{i}.move"
> > >
> > > # The output of SEEK will have ffprobe results at timestamps like this
> > > (with K being keyframe and D being negative):
> > > # [0.000000,K_ -0.066667,_D -0.100000,_D -0.033333,_D 0.135000,__
> > > 0.066667,__ 0.033333,__ 0.100000,__ 0.285000,__ ...
> > > # ... 0.826667,__ 0.910000,__ 1.076667,K_ 1.035000,__ 0.993333,__
> > > 1.243333,__ 1.160000,__]
> > >
> > > # Notice that 2.135000 - 1.068333 = 1.076667 as this is segment_2.mov;
> > > and timestamps are reset to start at zero.
> > >
> > > This command TRIMs each SEGMENT and does successfully fulfill the
> > > concatenation requirement; but is very, very slow (0.25x real time)
> > >
> > > # cmd = "ffmpeg -i segment_2.mov -vf trim=0:1.076667
> segment_2_trim.mov"
> > > #
> > > # The output of TRIM takes 4-5 seconds despite being a 1.07 second
> > > clip: re-encoding is always slow. But, it is correct.
> > > # [0.000000,K_ 0.033333,__ 0.016667,__ 0.008333,__ 0.025000,__
> > > 0.066667,__ 0.050000,__ 0.041667,__ ...
> > > # ... 1.016667,__ 1.058333,__ 1.041667,__ 1.033333,__ 1.050000,__]
> > >
> > > This python code launches CONCAT:
> > >
> > > # s = time.time()
> > > # tmp = tempfile.NamedTemporaryFile(delete=False)
> > > # relpath = os.path.relpath(os.getcwd(), tmp.name)
> > > # with open(tmp.name, "w") as f:
> > > #     [f.write(f"file '{relpath}/segment_{i}.mov'\n") for i in
> range(28)]
> > > # cmd = f"ffmpeg -y -f concat -safe 0 -i {tmp.name} -c copy
> > > concatenated.mov"
> > > # subprocess.run(cmd.split(), shell=False, cwd=os.getcwd())
> > > # print(time.time()-s)
> > >
> > > [this next one does not work: it is intended to create keyframes every
> > > one second. though this entire avenue may be unnecessary.]
> > >
> > > # cmd = f"ffmpeg -y -i IMG_9209.MOV -force_key_frames
> > > expr:gte(t,n_forced*1) -c copy keyframes.mov"
> > >
> > >
> > > Thank you.
> > > DB
> > >
> > >
> >
> > >
> > >
> > >
> > > > z!
> > > > _______________________________________________
> > > > ffmpeg-user mailing list
> > > > ffmpeg-user at ffmpeg.org
> > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> > > >
> > > > To unsubscribe, visit link above, or email
> > > > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
> > > >
> > > _______________________________________________
> > > ffmpeg-user mailing list
> > > ffmpeg-user at ffmpeg.org
> > > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> > >
> > > To unsubscribe, visit link above, or email
> > > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
> > >
> > _______________________________________________
> > ffmpeg-user mailing list
> > ffmpeg-user at ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
> >
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
>