[MPlayer-dev-eng] Some thoughts on telecine artifacts

[Tim U.]

     I don't know for sure what causes telecine artifacts in NTSC
DVDs, but here is my guess.  The film was first converted to MPEG-2
somehow.  This MPEG-2 stream probably contained 24 FPS of hard-coded
frames, and in the picture headers the repeat_first_field flag was
used so that a DVD player can generate 29.97 FPS to send to an NTSC
television.

     Then they probably wanted to do some editing.  The MPEG-2
editing software they used is probably the source of the artifacts.
They select some scenes to be cut out and start encoding the final
MPEG-2 stream.  Most of the original can be simply copied from the
input stream to the output stream, but the cutting points don't
necessarily occur on I-frames (Intra-coded frames which, IIRC, are
the ones which contain the entire picture as opposed to P-frames
which take most of the picture from surrounding frames).  So some
groups of pictures must be decoded to extract the frames which belong
in the scene.

     The decoder probably used the repeat_first_field flag and
produced 29.97 FPS with a 3, 2, 3, 2, etc sequence (3 normal frames,
2 frames with redundant fields).  So the first or last frame of a
scene could be any one of the five frames in a 3:2 sequence, and when
the scenes are concatenated, it is very possible to have more than 3
normal frames in a row, or more than 2 frames with redundant fields
in a row.  This will cause the IVTC algorithms used in mpeg2avi and
VirtualDub to fail.

     If one of a pair of 2 frames with redundant fields is the first
or last in a scene, then one field of the original film frame is
unrecoverable.  Otherwise it is possible to recover the original
frames by finding the first and last frames of the scenes, figuring
out the right IVTC offset to start at, and encoding each of these
scenes seperately.  I am attempting to do this with "The Saint" using
DVD2AVI->VFAPI->VirtualDub.  Currently I have 45032/208940 input
(29.97 FPS) frames completed.  As you might assume, this is NOT fun.
Especially since I'm stuck in Windows for hours on end.. maybe I
should try WINE.

     I own 13 DVDs so far, and 3 of them have significant telecine
artifacts when I play them with mplayer or encode them with mencoder.
They are "The Saint", "War Games", and "American Pie 2".  The Saint
is by far the worst, American Pie 2 has telecine artifacts during
about the last 1/3rd of the movie, it might be as bad or worse than
The Saint to fix if it has as many changes in the 3:2 sequence, but I
haven't looked at it closely yet (still working on The Saint).

     I don't know if there is anything that MPlayer can do to deal
with this.  Maybe if it notices a change from 24 FPS of coded frames
to 29.97 FPS of coded frames and vice versa it could display a
warning.  This wouldn't be a solution, but would make it a little
easier, if I had the frame numbers where these changes occur, to
isolate the hard-coded 29.97 FPS sequences in VirtualDub.  The 24 FPS
segments could be encoded without worrying about there being a change
in the 3:2 sequence somewhere in the middle.

     I think I should be able to add this little printf() myself, but
I'm not sure that the frame numbers displayed would coincide with
the frames numbers displayed in VirtualDub.  Don't worry about this
though, I'll figure it out, no sense in spending your time on it.

     Once the filter framework for mencoder and/or mplayer that I've
read about here on mplayer-dev-eng is ready, would it be useful to
have an IVTC filter?  A filter which would turn on when hard-coded
29.97 FPS segments are encountered, and off during 24 FPS segments.
The only problems would be detecting the IVTC offset at the beginning
of the segment, and dealing with changes in the 3:2 sequence in the
middle of a 29.97 FPS segment.  Maybe having a file which would
contain, one per line, a frame number and the IVTC offset at that
frame.  It could read a frame number and IVTC offset pair, and when
that frame is decoded, switch the IVTC algorithm to that offset.  I
would be willing to attempt to write this once the filter framework
is ready and once I have learned how it works.

     I am interested to hear any comments anyone may have on this 
issue.

   - Tim U.