[FFmpeg-devel] additive frame change detection algorithm

[Dave Hess]

I'm working on a post-process that will create thumbnails of a computer
screen capture.  Is there a frame change detection algorithm similar to
"scene" that will detect changes from a previously detected frame or from x
frames prior?  The current scene algorithm doesn't detect subtle changes in
frames when fade transitions are used in slide decks or when an instructor
is annotating on a slide.  Ultimately, I'd like the thumbnail to be created
at x frames forward of a detection so I'm not creating a thumbnail in
mid-transition.  That, I think, can be done outside of the detection
algorithm, so I'll concentrate on only the detection for now.  Correct me if
I'm wrong on that account.

 

The command line might look like the following where the algorithm is passed
two parameters (ref_frame, threshold) and returns either true (1) or false
(0) if the threshold is met. 

 

ffmpeg -i screen_capture_video.mp4 -vf "select='frame_change(ref_frame,
threshold)'" -vsync vfr thumbnail%03d.jpg

 

int ref_frame

0 = reference frame is previously selected frame (the last frame which the
algorithm returned true)

1 = reference frame is 1 frame prior

...

30 = reference frame is 30 frames prior (may need to place a max value to
minimize frames in memory during process)

 

double threshold

0 = minimum change detected

1 = maximum change detected

 

It looks like most of the work is already done in the scene algorithm.  My
proposed algorithm changes the implementation a little bit by passing
parameters and returning only true or false.  I live in the Windows world
and I don't have the expertise to implement this.  I looked at the code in
the get_scene_score function in the f_select.c file, but I quickly realized
I'm out of my league in terms of writing this myself.  I'm not even sure I'm
looking in the right place.  Any help or advice?

 

Dave Hess