[FFmpeg-devel] [FFmpeg-devel, RFC] lavfi: add opencl tonemap filter.

[Niklas Haas]

> The way my scene change detection heuristic works like this: I trigger a
> scene change (and therefore discard the frame averaging buffer) if the
> distance between the current frame average brightness and the current
> running average exceeds a threshold value, that threshold being (by
> default) 20 cd/m².
> 
> So we can divide the failures of this algorithm into two categories:
> 
> 1. False negative (scene change without resetting the buffer): This can
>    only happen if there was an actual scene change but the average
>    brightness change did not exceed 20 cd/m², i.e. the scenes are
>    similar in brightness. I consider this a fairly harmful failure
>    because that also means there's no visual discontinuity since the
>    scenes are so similar to begin with.
> 
> 2. False positive (buffer got reset without a scene change). This is the
>    more worrying failure of the algorithm, since it can happen in the
>    middle of a scene (e.g. as the result of a bright flash of light
>    on-screen), which will manifest itself as a sudden decrease in the
>    total frame brightness coinciding with the new source of light. (Or
>    vice versa, a sudden increase in brightness coinciding with the
>    sudden absence of a light source).
> 
> The scene change threshold is a trade-off. Lowering the value decreases
> the likelihood of #1 but increases the likelihood of #2. Increasing the
> value decreases the likelihood of #2, but increases the likelihood (and
> apparent effect) of #1.
> 
> If you want to optimize or improve the algorithm, the case #2 is the one
> I would be most interested in, i.e. reducing the rate of false
> positives. This can surely be done in a smarter way, e.g. by comparing
> more than just the scene average but also other easily obtained metrics.
> 
> If you have access to low-level frame information, you could do
> something like increasing the threshold for non-keyframes significantly,
> since keyframes are likely to coincide with scene changes or cuts this
> might help the algorithm out.

Thinking about this logic again, I came to realize that a different
strategy might be to check instead for a minimum threshold brightness
difference in a critical number of different areas of the screen. This
way, a very bright light source appearing or becoming occluded in one
local part of the frame will not trigger a scene change, while a sudden
change in brightness of a large part of the frame will.

I will try it if I get the opportunity to.