[FFmpeg-devel] The patch series about premultiplied alpha

[Nicolas George]

Nicolas George (HE12025-08-02):
> - The patch series lacks transverse user documentation.
> 
> - The patch series increases the risk of data corruption due to user
>   negligence and should include at least some guardrails against that.

Here is the situation before this patch series:

- All alpha is supposed to be straight by default.

- A few niche decoders (EXR, JPEG XL) would silently output
  premultiplied alpha.

- A few niche encoders (the same) would silently assume their input is
  premultiplied alpha.

- There is a pair of filters premultiply / unpremultiply.

- The overlay filter supports premultiplied alpha if enabled with an
  undocumented option.

- There is no protection at all: if a command connects input with one
  kind of alpha with out expecting the other kind at any point, it will
  produce invalid output without so much as a warning.

- There is barely any documentation at all. The filters are documented
  for what they do and that is all.

(IIRC, I would have fought the inclusion of the filters, but with Paul
one must choose one's battles.)

This patch series makes support for premultiplied alpha official.
Ideally, official support should include transparent automatic
conversion, but that would require too much work. But I think that
before inclusion it must fix the two worst problems:

- lack of documentation;

- risk of creating invalid output without any warning,

considering that the work to achieve it is a moderate fraction of the
work already invested in the patch series.

Documentation is easy: just write a few paragraphs to explain what
premultiplied alpha is, where it likely to occur (what did motivate the
writing of this series?) and how to deal with it (links to the
conversion filters).

Protection against creating invalid output, now. As I mentioned above,
full negotiation and automatic conversion, which would be ideal, is too
much work and therefore too much to ask. Niklas agree on that.

On the other hand, protection, i.e. emitting an error and letting the
user deal with it, is not much work:

- add a flag to filters that support premultiplied input;

- add a check for that flag in the framework.

In total, maybe two lines in a header, four lines in framework code and
one line per filter that will not damage the output. Similar work has
already be done for encoders.

Such limited work would make the feature much less dangerous for users,
it should be done before the feature reaches the public.

Regards,

-- 
  Nicolas George