[FFmpeg-devel] libopenjpeg decoder not correctly setting the pixel format for cinema JP2K wrapped MXF

[Rémi Achard]

> Do you have a sample that does not work with the native decoder?

According to my tests, the native decoder detect pixel format just fine,
I'm talking about the libopenjpeg wrapper here (ie. libopenjpegdec.c).

As for sample materials, I guess you can check this link
https://github.com/Ymagis/ClairMeta_Data

Specifically, we made DCP in various configurations with MXF video track
that last 1sec (minimum allowed by the SMPTE), let me know if that works
for you.
* 2D Interop MXF
https://github.com/Ymagis/ClairMeta_Data/blob/master/DCP/ECL-SET/ECL01-SINGLE-CPL_TST_S_EN-XX_UK-U_71_2K_DI_20171218_ECL_IOP_OV/ECL01-SINGLE-CPL_TST_S_EN-XX_UK-U_71_2K_DI_20171218_ECL_IOP_OV_01.mxf
* 3D SMPTE MXF
https://github.com/Ymagis/ClairMeta_Data/blob/master/DCP/ECL-SET/ECL09-SINGLE-CPL_TST_S_EN-XX_UK-U_51-ATMOS_2K_DI_20171220_ECL_SMPTE_OV/ECL-SINGLE-CPL_TST_S_EN-XX_UK-U_51_2K_DI_20171220_ECL_SMPTE_OV_01.mxf

> If you want the IS_KLV_KEY() check to be more lenient then you should
> make mxf_jp2k_rsiz[] shorter, not replace the end of it with NULs

This UL come straight from SMPTE ST 422M, not sure what you mean but I
didn't replace it with arbitrary NULs values.


Looking at mxfdec.c some more, I found that the pixel format is
supposed to be passed to the AVStream codecpar later on, but there is
some issues with that, that I don't found a solution yet:

* The JPEG2000PictureSubDescriptor live in it's own "MetadatSet" and
the pixel_format assigned is from a different Descriptor structure
than the one used for eg. width and height parameters
* Currently the pixel_format is only passed on for RAWVIDEO codec, so
it would require doing that for JP2K and potentially others as well


Le dim. 30 août 2020 à 18:22, Rémi Achard <remiachard at gmail.com> a écrit :

> Hi,
>
> As you are probably aware, the libopenjpeg decoder is not able to
> interpret cinema jp2k mxf correctly, the pixel format being assigned as
> rgb48 instead of xyz12 as it should. Note that ffmpeg native jp2k decoder
> parse the RSIZ descriptor from the jp2k bitstream and is able to correctly
> assign the pixel format, but libopenjpeg currently don't read the
> RSIZ marker so that ffmpeg wrapper has no simple mean of doing this at the
> moment (a pull request on libopenjpeg is still in progress to try to fix
> that).
>
> Someone recently pointed me to this commit from 2015
> https://github.com/FFmpeg/FFmpeg/commit/2af260e3a85ef2a9fadcac4f4fa652cee53e591e .
> According to my tests, it doesn't work because of a wrong RSIZ key and
> missing JP2KEssenceSubDescriptor lookup group. I made a patch I made to fix
> that, mxfdec is now able to correctly assign the descriptor->pix_fmt field.
> Note that these keys can be found in SMPTE ST 429-4 or discovered through
> asdcplib with asdcp-info command line tool parsing a cinema mxf video track.
>
> Then, even with that patch, the pixel format detected by the demuxer is
> not communicated to the decoder, as far as I'm aware. Which makes me
> wonder, what was the original goal of that commit and what I'm missing to
> communicate this information to libopenjpeg decoder ?
>
> diff --git a/libavformat/mxfdec.c b/libavformat/mxfdec.c
> index 6f6e8d586c..cd85fbedff 100644
> --- a/libavformat/mxfdec.c
> +++ b/libavformat/mxfdec.c
> @@ -325,7 +325,7 @@ static const uint8_t mxf_encrypted_essence_container[]
>     = { 0x06,0x0e,0x2b,0x
>  static const uint8_t mxf_random_index_pack_key[]           = {
> 0x06,0x0e,0x2b,0x34,0x02,0x05,0x01,0x01,0x0d,0x01,0x02,0x01,0x01,0x11,0x01,0x00
> };
>  static const uint8_t mxf_sony_mpeg4_extradata[]            = {
> 0x06,0x0e,0x2b,0x34,0x04,0x01,0x01,0x01,0x0e,0x06,0x06,0x02,0x02,0x01,0x00,0x00
> };
>  static const uint8_t mxf_avid_project_name[]               = {
> 0xa5,0xfb,0x7b,0x25,0xf6,0x15,0x94,0xb9,0x62,0xfc,0x37,0x17,0x49,0x2d,0x42,0xbf
> };
> -static const uint8_t mxf_jp2k_rsiz[]                       = {
> 0x06,0x0e,0x2b,0x34,0x02,0x05,0x01,0x01,0x0d,0x01,0x02,0x01,0x01,0x02,0x01,0x00
> };
> +static const uint8_t mxf_jp2k_rsiz[]                       = {
> 0x06,0x0e,0x2b,0x34,0x01,0x01,0x01,0x0a,0x04,0x01,0x06,0x03,0x01,0x00,0x00,0x00
> };
>  static const uint8_t mxf_indirect_value_utf16le[]          = {
> 0x4c,0x00,0x02,0x10,0x01,0x00,0x00,0x00,0x00,0x06,0x0e,0x2b,0x34,0x01,0x04,0x01,0x01
> };
>  static const uint8_t mxf_indirect_value_utf16be[]          = {
> 0x42,0x01,0x10,0x02,0x00,0x00,0x00,0x00,0x00,0x06,0x0e,0x2b,0x34,0x01,0x04,0x01,0x01
> };
>
> @@ -2746,6 +2746,7 @@ static const MXFMetadataReadTableEntry
> mxf_metadata_read_table[] = {
>      { {
> 0x06,0x0e,0x2b,0x34,0x02,0x53,0x01,0x01,0x0d,0x01,0x01,0x01,0x01,0x01,0x42,0x00
> }, mxf_read_generic_descriptor, sizeof(MXFDescriptor), Descriptor }, /*
> Generic Sound */
>      { {
> 0x06,0x0e,0x2b,0x34,0x02,0x53,0x01,0x01,0x0d,0x01,0x01,0x01,0x01,0x01,0x28,0x00
> }, mxf_read_generic_descriptor, sizeof(MXFDescriptor), Descriptor }, /*
> CDCI */
>      { {
> 0x06,0x0e,0x2b,0x34,0x02,0x53,0x01,0x01,0x0d,0x01,0x01,0x01,0x01,0x01,0x29,0x00
> }, mxf_read_generic_descriptor, sizeof(MXFDescriptor), Descriptor }, /*
> RGBA */
> +    { {
> 0x06,0x0e,0x2b,0x34,0x02,0x53,0x01,0x01,0x0d,0x01,0x01,0x01,0x01,0x01,0x5a,0x00
> }, mxf_read_generic_descriptor, sizeof(MXFDescriptor), Descriptor }, /*
> JPEG2000PictureSubDescriptor */
>      { {
> 0x06,0x0e,0x2b,0x34,0x02,0x53,0x01,0x01,0x0d,0x01,0x01,0x01,0x01,0x01,0x48,0x00
> }, mxf_read_generic_descriptor, sizeof(MXFDescriptor), Descriptor }, /*
> Wave */
>      { {
> 0x06,0x0e,0x2b,0x34,0x02,0x53,0x01,0x01,0x0d,0x01,0x01,0x01,0x01,0x01,0x47,0x00
> }, mxf_read_generic_descriptor, sizeof(MXFDescriptor), Descriptor }, /*
> AES3 */
>      { {
> 0x06,0x0e,0x2b,0x34,0x02,0x53,0x01,0x01,0x0d,0x01,0x01,0x01,0x01,0x01,0x51,0x00
> }, mxf_read_generic_descriptor, sizeof(MXFDescriptor), Descriptor }, /*
> MPEG2VideoDescriptor */
>
>