[FFmpeg-devel] [PATCH] swscale: use 16-bit intermediate precision for RGB/XYZ conversion

[Niklas Haas]

On Tue, 17 Dec 2024 00:36:13 +0100 Michael Niedermayer <michael at niedermayer.cc> wrote:
> Hi
>
> On Mon, Dec 16, 2024 at 02:56:07PM +0100, Niklas Haas wrote:
> > From: Niklas Haas <git at haasn.dev>
> >
> > The current logic uses 12-bit linear light math, which is woefully insufficient
> > and leads to nasty postarization artifacts. This patch simply switches the
> > internal logic to 16-bit precision.
> >
> > This raises the memory requirement of these tables from 32 kB to 272 kB.
> >
> > Fixes: ticket 4829
> > Signed-off-by: Niklas Haas <git at haasn.dev>
> > Sponsored-by: Sovereign Tech Fund
> > ---
> >  libswscale/swscale.c          | 16 ++++++++--------
> >  libswscale/swscale_internal.h |  8 ++++----
> >  libswscale/utils.c            | 19 ++++++++++++-------
> >  3 files changed, 24 insertions(+), 19 deletions(-)
>
> breaks fate

I double-checked and can confirm that these are not regressions, but reflections
of the improvement in conversion accuracy. I will squash the FATE changes into
the same commit before merging.

>
> TEST    filter-pixdesc-xyz12be
> --- ./tests/ref/fate/filter-pixdesc-xyz12be	2024-11-29 20:32:49.757718063 +0100
> +++ tests/data/fate/filter-pixdesc-xyz12be	2024-12-17 00:34:49.832342280 +0100
> @@ -1 +1 @@
> -pixdesc-xyz12be     4ec824668b9753e26c1bccffca866e27
> +pixdesc-xyz12be     1508a33dea936c45d9ee13f7743af00d
> Test filter-pixdesc-xyz12be failed. Look at tests/data/fate/filter-pixdesc-xyz12be.err for details.
> make: *** [tests/Makefile:311: fate-filter-pixdesc-xyz12be] Error 1
>
> thx
>
> [...]
> --
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> Breaking DRM is a little like attempting to break through a door even
> though the window is wide open and the only thing in the house is a bunch
> of things you dont want and which you would get tomorrow for free anyway
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".