[FFmpeg-devel] Support MXF separate fields layout with interlaced j2k

[Matthieu Bouron]

On Sat, Feb 16, 2013 at 05:44:49PM +0100, Matthieu Bouron wrote:
> On Sat, Feb 16, 2013 at 09:23:05AM -0500, Dave Rice wrote:
> > Hi Matthieu,
> > 
> > On Feb 16, 2013, at 7:52 AM, Matthieu Bouron <matthieu.bouron at gmail.com> wrote:
> > 
> > > This serie of patches tends to fix ticket #1102 by supporting mxf
> > > separate fields layout with jpeg2000 codec (libopenjpeg only).
> > > 
> > > Last patch introduce the full frame reconstuction within the
> > > libopenjpeg decoder if field_order is not unknown nor progressive.
> > > 
> > > The problem with this implementation is that the frame threading must be
> > > disabled manually by the user, if not, no frame is decoded and an
> > > error message is displayed to the user.
> > > 
> > > Afaik, the same result might be achieved with the tinterlace filter without
> > > this extra code however it feels more confortable to me to have a "native"
> > > support.
> > 
> > I'm very happy to see this support added and have tested it. I was unable to successfully apply your patches to git master and had to do it manually but got it running. I then tested the patch with your sample from Amberfin and then another sample that uses field-based jpeg2000 is MXF from the SAMMA system (sample here: https://ia601204.us.archive.org/10/items/sample_losslessj2k_mxf/test-jpeg2000-color-wsound1.mxf).
> > 
> > I agree with you that the adding this as native support is better than requiring the user to detect it and apply the tinterlace filter.
> > 
> > I note that with ntsc2.mxf and ffplay I get this:
> > 
> > ./ffplay -threads 1 ~/Desktop/ntsc2.mxf 
> > ffplay version N-50007-gbb29ee6 Copyright (c) 2003-2013 the FFmpeg developers
> >   built on Feb 16 2013 08:39:59 with llvm-gcc 4.2.1 (LLVM build 2336.11.00)
> >   configuration: --enable-libopenjpeg
> >   libavutil      52. 17.101 / 52. 17.101
> >   libavcodec     54. 91.103 / 54. 91.103
> >   libavformat    54. 63.100 / 54. 63.100
> >   libavdevice    54.  3.103 / 54.  3.103
> >   libavfilter     3. 38.100 /  3. 38.100
> >   libswscale      2.  2.100 /  2.  2.100
> >   libswresample   0. 17.102 /  0. 17.102
> > [mxf @ 0x7f7f9201a600] broken or empty indexaq=    0KB vq=    0KB sq=    0B f=0/0   
> > Input #0, mxf, from '/Users/davidrice/Desktop/ntsc2.mxf':=    0KB sq=    0B f=0/0   
> >   Metadata:
> >     timecode        : 02:57:45;02
> >   Duration: 00:00:01.53, start: 0.000000, bitrate: 78647 kb/s
> >     Stream #0:0: Video: jpeg2000, yuv422p10le, 720x486, 29.97 fps, 29.97 tbr, 29.97 tbn, 29.97 tbc
> >    1.44 A-V:  0.000 fd=  11 aq=    0KB vq= 1919KB sq=    0B f=0/0   =    0B f=0/0
> > 
> > and the file plays back at 29.97 fps.
> > 
> > With test-jpeg2000-color-wsound1.mxf I get this:
> > 
> > ./ffplay -threads 1 ~/Desktop/test-jpeg2000-color-wsound1.mxf 
> > ffplay version N-50007-gbb29ee6 Copyright (c) 2003-2013 the FFmpeg developers
> >   built on Feb 16 2013 08:39:59 with llvm-gcc 4.2.1 (LLVM build 2336.11.00)
> >   configuration: --enable-libopenjpeg
> >   libavutil      52. 17.101 / 52. 17.101
> >   libavcodec     54. 91.103 / 54. 91.103
> >   libavformat    54. 63.100 / 54. 63.100
> >   libavdevice    54.  3.103 / 54.  3.103
> >   libavfilter     3. 38.100 /  3. 38.100
> >   libswscale      2.  2.100 /  2.  2.100
> >   libswresample   0. 17.102 /  0. 17.102
> > [mxf @ 0x7fd85380f200] broken or empty indexaq=    0KB vq=    0KB sq=    0B f=0/0   
> > [mxf @ 0x7fd85380f200] error getting stream index 67174400
> >     Last message repeated 41 times0 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0   
> > [mxf @ 0x7fd85380f200] Estimating duration from bitrate, this may be inaccurate
> > Input #0, mxf, from '/Users/davidrice/Desktop/test-jpeg2000-color-wsound1.mxf':
> >   Metadata:
> >     timecode        : 01:03:34:21
> >   Duration: 00:35:01.75, start: 0.000000, bitrate: 3071 kb/s
> >     Stream #0:0: Video: jpeg2000, yuv422p, 720x486, 59.94 fps, 59.94 tbr, 59.94 tbn, 59.94 tbc
> >     Stream #0:1: Audio: pcm_s16le, 48000 Hz, 4 channels, s16, 3072 kb/s
> >    0.47 A-V:  0.037 fd=   5 aq=   31KB vq= 1104KB sq=    0B f=0/0
> > 
> > The video plays back at 59.94 fps which is the field rate rather than playing at the frame rate 29.97. When I try to transcode test-jpeg2000-color-wsound1.mxf to another output I get a 59.94 fps output as every frame is duplicated. 
> > 
> > In "CDCI Essence Descriptor", if "Interlace or Progressive layout" (0x320C) is 1, then the encoding is field per field rather than frame per frame, so for a natural presentation you'd need to suppose both that:
> > - Height is the height of a field, not a frame
> 
> I agree.
> 
> > - Frame rate is field rate
> > 
> 
> Is this behaviour described somewhere ?
> Following this logic, the file associated with ticket #1102 would play at
> 15000/1001 which do not feel natural to me.
> 
> Wild guess: the frame duplication occurs because i did not correct the frame
> pts in the decoder.

The frame duplication occurs because the decoder outputs one frame every
two video packets. The outputed frame takes the pts from the second packet
causing ffmpeg to duplicate one frame. A temporary fix is to use -vsync 0.

The only ugly solution i see right now (and it seems to work), is to generate
the ptses for the interlaced + j2k case so that the pair of video packet
needed to reconstruct a given frame would have the same pts.

pkt0.pts = 0, pkt1.pts = 0 -> decoded_frame.pkt_pts = 0
pkt2.pts = 1, pkt3.pts = 1 -> decoded_frame.pkt_pts = 1
...

Better ideas welcomed :)

Regards,
Matthieu