[FFmpeg-devel] [PATCH 00/27] Metadata handling in CBS

[Mark Thompson]

Happy New Year!

The fun part here is:

AVMasteringDisplayMetadata mdm = { ... };
AVContentLightMetadata clm = { ... };
ff_cbs_insert_metadata(cbc, fragment, CBS_METADATA_MASTERING_DISPLAY,   &mdm);
ff_cbs_insert_metadata(cbc, fragment, CBS_METADATA_CONTENT_LIGHT_LEVEL, &clm);

which does whatever is needed to splice that metadata into the given bitstream fragment.

It's implemented for H.264, H.265 and AV1 here.  Extension to the common parts of H.266 is trivial (just add to the type table).

The metadata bitstream filters are extended to support that as well from side data.  That's only library-usable, though - there is no ffmpeg-the-program interface for including the side data in the first place.  (From discussion on IRC a few days ago there was interest in adding ffmpeg support for that along with automatic BSF insertion?)

Not particularly well tested, since much of this was written in the last few days (though some parts like the common BSF implementation have been sent before).

- Mark


Mark Thompson (27):
  cbs_h2645: Merge SEI messages in common between codecs
  h264_metadata_bsf: Move SEI user data parsing to init time
  cbs_h2645: Merge SEI message handling in common between codecs
  cbs_sei: Remove restrictions on MDCV values
  h264_metadata_bsf: Refactor the filter function into smaller parts
  h264_metadata_bsf: Improve interpretation of input display matrices
  cbs: Implement common parts of cbs-based bitstream filters separately
  h264_metadata_bsf: Use common cbs bsf implementation
  h265_metadata_bsf: Use common cbs bsf implementation
  mpeg2_metadata_bsf: Use common cbs bsf implementation
  vp9_metadata_bsf: Use common cbs bsf implementation
  av1_metadata_bsf: Use common cbs bsf implementation
  h264_redundant_pps_bsf: Use common cbs bsf implementation
  cbs: Add interface for manipulating metadata in fragments
  cbs_sei: Implement metadata manipulation for SEI codecs
  cbs_sei: Implement fill and extract for HDR SEI messages
  cbs_h264: Implement fill and extract for display orientation SEI
    message
  cbs_av1: Implement metadata manipulation
  cbs_av1: Support manipulating HDR CLL and MDCV metadata
  cbs: Add interface to allow BSFs easy access to metadata manipulation
  h264_metadata_bsf: Simplify display orientation handling
  h264_metadata_bsf: Support HDR metadata
  h265_metadata_bsf: Support HDR metadata
  av1_metadata_bsf: Support HDR metadata
  vaapi_encode: Unify SEI option mask enum
  vaapi_encode_h265: Ensure that HDR metadata is included with seek
    points
  vaapi_encode_h264: Support HDR metadata

 doc/bitstream_filters.texi            |  51 +++
 libavcodec/Makefile                   |   6 +-
 libavcodec/av1_metadata_bsf.c         | 198 ++-------
 libavcodec/cbs_av1.c                  | 233 +++++++++++
 libavcodec/cbs_bsf.c                  | 234 +++++++++++
 libavcodec/cbs_bsf.h                  | 106 +++++
 libavcodec/cbs_h264.h                 |  79 +---
 libavcodec/cbs_h2645.c                | 494 ++++++++++++++++------
 libavcodec/cbs_h264_syntax_template.c | 271 +-----------
 libavcodec/cbs_h265.h                 |  66 +--
 libavcodec/cbs_h265_syntax_template.c | 370 ++---------------
 libavcodec/cbs_internal.h             |  17 +
 libavcodec/cbs_metadata.c             | 107 +++++
 libavcodec/cbs_metadata.h             |  94 +++++
 libavcodec/cbs_sei.c                  | 469 +++++++++++++++++++++
 libavcodec/cbs_sei.h                  | 350 ++++++++++++++++
 libavcodec/cbs_sei_syntax_template.c  | 307 ++++++++++++++
 libavcodec/h264_metadata_bsf.c        | 569 ++++++++++----------------
 libavcodec/h264_redundant_pps_bsf.c   |  99 +----
 libavcodec/h265_metadata_bsf.c        | 213 +++-------
 libavcodec/mpeg2_metadata_bsf.c       |  94 +----
 libavcodec/vaapi_encode.h             |   8 +
 libavcodec/vaapi_encode_h264.c        | 125 ++++--
 libavcodec/vaapi_encode_h265.c        | 123 ++----
 libavcodec/vp9_metadata_bsf.c         |  61 +--
 25 files changed, 2859 insertions(+), 1885 deletions(-)
 create mode 100644 libavcodec/cbs_bsf.c
 create mode 100644 libavcodec/cbs_bsf.h
 create mode 100644 libavcodec/cbs_metadata.c
 create mode 100644 libavcodec/cbs_metadata.h
 create mode 100644 libavcodec/cbs_sei.c
 create mode 100644 libavcodec/cbs_sei.h
 create mode 100644 libavcodec/cbs_sei_syntax_template.c

-- 
2.29.2