[FFmpeg-devel] Rework color quantization in palette{gen,use}

[Clément Bœsch]

Hi,

This patchset essentially fixes a few core problems in these filters and
switches to a perceptual model.

I've generated a report for each key commit on this (temporary) page:
http://big.pkh.me/pal/ (warning: heavy page, ~500M; I did try to add some lazy
loading of the images but I'm not sure it's actually working as expected).

It is easy for me to add specific samples and re-run the whole thing, so feel
free to suggest one.

A summary on a few important observed differences can be found on the page, but
I'm duplicating it here for the record:

  - Current: current state on master
  - Paletteuse Perceptual
      + same palette but better selection: instead of rgb triplet distance, it uses a
        colorspace designed for uniform perceptual color differences (OkLab)
      + overall impact not that visible immediately, but it will make sure the
        palette is used the best way possible, meaning improvements to palettegen
        will be honored
      + observations (with no dither):
          * file02 (rooftops) in max_colors=8 or 16: sky pattern is more stable
          * file06 (parrot) in max_colors=8: better color for the parrot beak
          * overall seems to favor slightly brighter colors in the currently
            offered palette
  - Palettegen Linear Average
      + sRGB colors are gamma encoded, averaging them naively is incorrect, we
        need to do that in linear space
      + observations (with no dither):
          * file00 (colorful drawing) in max_colors=8: contrast and color skin
            look better
          * file07 (abstract flower) in max_color=128 or 256: this picture
            composed of 1M different colors in the source is now more balanced
            (better spreading of the colors)
  - Palettegen Perceptual
      + similar to the paletteuse perceptual, we use OkLab for the color distance
      + observations (with no dither):
          * file07 (abstract flower): in max_colors=128 or 256 we can see the
            picture offering a much finer grain.
  - Palettegen Variance per axis
      + When deciding on spliting a box along an axis, instead of picking the
        longest one, we pick the one with the most color variance
      + Not that much impact
  
  Overall, the most brutal change is probably in file07 between current and last,
  256 colors no dither in particular.

Finally, I do believe a lot of other color filters could at least benefit from
fixing their gamma handling (I know I'm guilty of this in various other
filters).

Regards,

-- 
Clément B.