[FFmpeg-devel] [RFC] Dynamic Hald CLUT

[Clément Bœsch]

Hi folks,

I'm on a new project based on the recently added 3D LUT filter, and I
would like some comments.

Basically, a 3D LUT is a RGB color map. In theory, with 8-bit, you would
have 16M colors (256*256*256), one axis for each component, each "node"
associated with a new RGB color. In practice, it's generally a 16*16*16 or
eventually 32*32*32 color map, and the missing colors are interpolated
using various methods. It also makes possible to store a 16-bit map
without an insane amount of memory requirement.

So this is the basic theory behind the 3D LUT. Though, most of the formats
used to store them are different, and it's a pain to support them
properly. Fortunately, someone came up with the idea to store those tables
not in a text based file, but simply in a picture. This is called Hald
CLUT¹

Apart from the format being natively supported in FFmpeg, it also makes
possible something pretty awesome: we can apply several color filters to
the image (with FFmpeg or any external tools), and then reuse the
resulting image to reproduce the effect.

I'm working on implementing this in the lut3d filter. Actually, it's
already done and available on my Github². But I'm not satisfied by it for
various reasons. The main obvious one is that it creates a lot of
pointless dependencies on lavfi to get that feature, but that's not the
only one.

What I would like to achieve is that instead of having the clut image file
passed as a parameter like lut3d=foobar.png, simply use a video stream
where the clut is picked. This has various benefits from the dependency
PoV, and the pixel format conversions, but moreover it also allows us to
make something extremely powerful: create Hald CLUT video streams. That
way, assuming we have such video stream, we could apply a complex color
mapping changing over time, without the need to recompute the colors with
the successive filters. It also makes possible to apply an external color
filter (imagine a sepia color mapping gradually changing for instance).

So, what needs to be done?

First step would be to be able to generate a CLUT stream. This is pretty
simple, we just need to add a source filter with an identity color frame
repeated overtime.

We could then imagine generating a Hald CLUT video stream with something
like:

  ffmpeg -f lavfi haldclutsrc,curves=vintage,negate,... -c:v ffv1 clut.nut

(and even adding various timeline information to make things interesting).

Then, we would be able to use the generated clut on video streams to
reproduce the effect:

  ffmpeg -i in.avi -i clut.nut -filter_complex '[0:0][1:0] haldclut' ...

OK, so this is the theory. Now in practice I'm facing a problem I don't
like very much: I need to reproduce the logic of -vf overlay to get a
meaningful synchronization between the input and the clut. And... I don't
feel like duplicating at all the insane complexity of the related code in
that filter.

So I'd like comment on someone familiar with that part of the overlay
filter, and suggestion on making things smooth. Note that it will likely
take me ages to make something sane out of this, especially since I don't
really want to work on that at all.

Anyway, any help to work on improving filters similar to overlay would be
very welcome (and I believe other filters such as blend could benefit from
it).

Feedback welcome,

Regards,

[1]: http://www.quelsolaar.com/technology/clut.html
[2]: https://github.com/ubitux/FFmpeg/compare/master...lut3d

-- 
Clément B.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 490 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20130523/a1ca1285/attachment.asc>