[FFmpeg-devel] SWS cleanup / SPI Funding Suggestion

[Niklas Haas]

On Sat, 14 Oct 2023 19:00:36 +0200 Michael Niedermayer <michael at niedermayer.cc> wrote:
> Well there are 2 further aspects with that.
> 
> The first one is bluntly put. If you dont understand the old code, then
> you probably are not qualified to write better code.
> People tend not to successfully improve things they dont understand.

I have a deep understanding of colorspaces and the necessary conversion
steps between them, and am also in a good position to integrate
libplacebo as a possible backend.

However, I do not have a good understanding of CPU/SIMD code, nor the
various swscale internals, beyond the cursory investigation I needed for
some recent swscale bugs I encountered. So I'll definitely need some
help along the way to fully understand those swscale internals.

> The 2nd issue is, ATM, i maintain swscale. If iam involved in the new
> effort and understand it either because of that or because it has some
> similarity then i can continue to maintain swscale. If its totally
> different and i was totally not involded then i also will not maintain
> it obviously.

I think it would be possible to join forces to the extent needed to
arrive at a satisfactory result. At the very least, I have very limited
experience working with "irregular" packed formats.

Obviously, my intent is not to blanket discard the swscale internals. It
has years and years of optimized kernels for various platforms just
lying around, wanting to be used. Hence my proposal of redesigning the
high-level logic, rather than the low-level details.

> This is something to be especially aware of in case the cleanup/new
> code would be done by someone who comes, does it and leaves. you
> could end up with nicer code thats then unmaintained.
> 
> PS: whats the real issue with sws ?
> it evolved out of a piece yuv->rgb converter from a video player.
> It evolved from that and stuff was added into it.
> This is a similar situation to why ffmpeg.c needed cleanup

Yes, it amounts to the usual disentangling of various special cases and
branches into one top-down control flow that knows about all of these
special cases and fast/slow paths to begin with.

My goal is to arrive at a place where we have one single code flow that
looks something like:

1. Settle the complete descriptions of the source and destination format/csp
2. Establish a list of operations to get from A to B, taking into
   account user settings
3. Determine and dispatch the best available functions for each operation

With the necessary code separation and/or layers of abstraction in place
to make this design manageable. In particular, steps 1 and 2 should be
expanded to include things like conversion between primaries, conversion
between HDR and SDR, conversion between YUV/RGB, and so on.

In particular, I also want to eventually add the ability to plug "Apply
a 3DLUT" in as a possible operation type for colorspace conversion,
probably by sharing the code that is already written for vf_lut3d.