[FFmpeg-devel] Towards YUVJ removal

[Michael Niedermayer]

On Fri, Dec 09, 2022 at 12:49:41PM +0100, Niklas Haas wrote:
> So, as was discussed at the last meeting, we should move towards
> removing YUVJ. I want to gather feedback on what appears to be to the
> major hurdle, and possible ways to solve it.
> 
> The basic major issue is how to handle the case of combining limited
> range input with an encoder for a format that only accepts full range
> data. The common case, for example, would be converting a frame from a
> typical video file to a JPEG:
> 
> $ ffmpeg -f lavfi -i smptebars -vframes 1 output.jpg
> 
> Currently, this works because the JPEG encoder only advertises YUVJ
> pixel formats, and therefore ffmpeg auto-inserts swscaler to convert
> from limited range to full range. But this depends on conflating color
> range and pixel formats, which is exactly what has been marked
> deprecated for an eternity.
> 
> Now, there are some solutions I can see for how to handle this case in
> a non-YUVJ world:
> 
> 1. Simply output an error, and rely on the user to insert a conversion
>    filter, e.g.:
> 
>    $ ffmpeg -f lavfi -i smptebars -vframes 1 output.jpg
>    error: inputs to jpeg encoder must be full range
> 
>    $ ffmpeg -f lavfi -i smptebars -vframes 1 -vf scale=out_range=jpeg output.jpg
>    ... works
> 
> 2. Have the JPEG encoder take care of range conversion internally, by
>    using sws to convert limited to full range.
> 
> 3. Allow filters to start exposing color space metadata as part of
>    filter negotiation, and then auto-insert swscaler whenever colorspace
>    conversion needs to happen as a result of filters not accepting the
>    corresponding color metadata. This would also allow more than just
>    conversion between limited/full range.
> 
> 4. Combine approach 1 with an encoder flag for "supports full range
>    only", and have ffmpeg.c auto-insert a scale filter as the last step
>    before the encoder if this flag is set (and input is limited range).
> 
> 1 would be the most explicit but would break any existing pipeline that
> includes conversion to jpeg, which is probably a very large number.
> 
> 2 would be the least work, but violates abstraction boundaries.
> 
> 3 would be the most work and is, IMO, of questionable gain.
> 
> 4 would be both explicit and not break existing workflows.

3 is nice but is hard as it draws filter negotiation in and that has
to do something even for non trivial filter networks.

4 with the complexity in jpeg as disscussed elsewhere in this thread
also may or may not be as clean as it sounds here

but both 3 and 4 with some amendment sound reasonable to me
another option would be to either improve 

A. general "supported value" information
a encoder should be able to tell the caller what it supports depending
on some parameter, for example depending on profile and level
or std compliance. This would mean implementing AVClass.query_ranges()
and using av_opt_query_ranges() IIRC


B. error reporting so that 
failures become machiene readable. 
aka, "this failed because color range is not X || std complicance is > Y"
the lib user could then retry by applying the change if that is within what
the usecase wants

Both A and B obvioulsy have a much broader use than YUVJ removal

thx


[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The real ebay dictionary, page 2
"100% positive feedback" - "All either got their money back or didnt complain"
"Best seller ever, very honest" - "Seller refunded buyer after failed scam"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20221209/7041d4d5/attachment.sig>