[FFmpeg-devel] Rework color quantization in palette{gen,use}

[Paul B Mahol]

On 11/5/22, Clément Bœsch <u at pkh.me> wrote:
> Hi,
>
> This patchset essentially fixes a few core problems in these filters and
> switches to a perceptual model.
>
> I've generated a report for each key commit on this (temporary) page:
> http://big.pkh.me/pal/ (warning: heavy page, ~500M; I did try to add some
> lazy
> loading of the images but I'm not sure it's actually working as expected).
>
> It is easy for me to add specific samples and re-run the whole thing, so
> feel
> free to suggest one.
>
> A summary on a few important observed differences can be found on the page,
> but
> I'm duplicating it here for the record:
>
>   - Current: current state on master
>   - Paletteuse Perceptual
>       + same palette but better selection: instead of rgb triplet distance,
> it uses a
>         colorspace designed for uniform perceptual color differences (OkLab)
>       + overall impact not that visible immediately, but it will make sure
> the
>         palette is used the best way possible, meaning improvements to
> palettegen
>         will be honored
>       + observations (with no dither):
>           * file02 (rooftops) in max_colors=8 or 16: sky pattern is more
> stable
>           * file06 (parrot) in max_colors=8: better color for the parrot
> beak
>           * overall seems to favor slightly brighter colors in the currently
>             offered palette
>   - Palettegen Linear Average
>       + sRGB colors are gamma encoded, averaging them naively is incorrect,
> we
>         need to do that in linear space
>       + observations (with no dither):
>           * file00 (colorful drawing) in max_colors=8: contrast and color
> skin
>             look better
>           * file07 (abstract flower) in max_color=128 or 256: this picture
>             composed of 1M different colors in the source is now more
> balanced
>             (better spreading of the colors)
>   - Palettegen Perceptual
>       + similar to the paletteuse perceptual, we use OkLab for the color
> distance
>       + observations (with no dither):
>           * file07 (abstract flower): in max_colors=128 or 256 we can see
> the
>             picture offering a much finer grain.
>   - Palettegen Variance per axis
>       + When deciding on spliting a box along an axis, instead of picking
> the
>         longest one, we pick the one with the most color variance
>       + Not that much impact
>
>   Overall, the most brutal change is probably in file07 between current and
> last,
>   256 colors no dither in particular.
>
> Finally, I do believe a lot of other color filters could at least benefit
> from
> fixing their gamma handling (I know I'm guilty of this in various other
> filters).

gamma handling depends not on pixel format but on metadata present in frame.

>
> Regards,
>
> --
> Clément B.
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>