[FFmpeg-devel] [PATCH 1/4] fftools/ffmpeg_enc: strip DOVI config record for AV1

[Niklas Haas]

On Thu, 21 Mar 2024 11:16:57 +0100 Anton Khirnov <anton at khirnov.net> wrote:
> Quoting Niklas Haas (2024-03-19 20:16:39)
> > From: Niklas Haas <git at haasn.dev>
> > 
> > AV1 streams don't use configuration records, so delete them when
> > encoding to AV1. Ideally this would be, as the comment suggests, handled
> > at the frame-level (and stripped by the av1 encoder), but given the
> > status quo of copying the packet-level data here directly, we should
> > definitely make an effort to strip it.
> > ---
> >  fftools/ffmpeg_enc.c | 25 ++++++++++++++-----------
> >  1 file changed, 14 insertions(+), 11 deletions(-)
> 
> I'm very much not a fan of having codec-specific code in ffmpeg CLI. It
> implies that every single caller must now be aware of this
> (undocumented?) interaction of this specific side data with this
> specific codec ID.

Note: This is an existing bug, not introduced by this series. This
series just makes it obvious. The status quo is that, beacuse of this
logic in ffmpeg_enc.c, we incorrectly forward dolby vision configuration
records when transcoding to AV1.

Or, indeed, when transcoding to *any* format - since current FFmpeg also
does not propagate dolby vision RPUs, we generate broken files pretty
much always when transcoding dolby vision. So we definitely need to
strip the metadata from the stream muxer *somewhere*. Where else comes
to mind?

This also gets into another topic I wanted to touch on, which is that
the presence of dynamic dolby vision metadata currently hinders the
ability of libavfilter to treat the video primaries/gamma as
a negotiable colorspace property (the way it is done currently for YUV
matrix/range). This is because when interpreted as such, DV metadata
fundamentally changes the colorspace of the incoming video stream.
Ideally we would like some way to negotiate DV metadata on the
query_formats() level.

Ideally, we'd want something like AVCOL_SPC_DOLBYVISION, but we can't
easily introduce that without breaking ISO/IEC 23091 compatibility..

-------
P.s. I accidentally sent a duplicate of this email from a different
(wrong) mail address, please ignore.