[MPlayer-dev-eng] Need help with A/V sync in new video filter

[Walter Belhaven]

Hi,

I'm trying to implement a filter that 'unpacks' mpg2 frames that have
their Repeat_First_Field flag set.  Here's an example from an actual
HD 1080i broadcast (INPUT) and how I'd like to transform it (OUTPUT):

      INPUT     |            OUTPUT
  Frame TFF RFF | Frame TFF RFF  (input fields)
  --------------+-----------------------------------
    1    1   0  |   1    1   0   (top=1, bot=1)
  --------------+-----------------------------------
    2    1   1  |   2    1   0   (top=2, bot=2)
  --------------+-----------------------------------
    3    0   0  |   3    1   0   (top=2, bot=3)
  --------------+-----------------------------------
    4    0   1  |   4    1   0   (top=3, bot=4)
                |   5    1   0   (top=4, bot=4)       <-- "extra"
  --------------+-----------------------------------
    5    1   0  |   6    1   0   (top=5, bot=5)

Note the consistent TFF=1 and RFF=0 of the output pattern.  The
reasons I need this are related to transcoding 1080i "video" (~60
unique fields per second) to NTSC DVD, without deinterlacing, pullup,
or anything else of the sort, and without having to fight seemingly
mysterious "field order changes" in the interlaced result.

The closest thing I can find to what I want is '-vf softpulldown', but
unlike that filter, I want mine to handle ALL cases of the RFF flag,
not just the "special case" of the "proper" 3:2 soft-telecine pattern.
(Softpulldown complains with lots of "Unexpected field flags" when
analyzing my 1080i source, and doesn't do what I want when it
encounters those 'unexpected' flags.)

After a lot of pain and experimentation, I've gotten close, but I have
a couple of remaining problems that I could use some help with.

First, the Video Filter section in tech/libmpcodecs.txt says, in
effect, thou shalt not pass mpi down the chain as-is, even if you
don't modify it.  Is this still considered appropriate advice?  I
found a handful of cases where implementors have ignored that, such as
in the very softpulldown I mentioned.  If my filter is going to pass a
PARTICULAR Frame as-is, can I just pass 'mpi' along?  What if I'm just
changing one of the bits in mpi->fields?  Or in both cases (no change
or just a flag change) should I really do this:

    dmpi = vf_get_image(vf->next, mpi->imgfmt,
       MP_IMGTYPE_EXPORT, 0, mpi->width, mpi->height);

    dmpi->planes[0] = mpi->planes[0];
    dmpi->stride[0] = mpi->stride[0];
    if (dmpi->flags&MP_IMGFLAG_PLANAR) {
        dmpi->planes[1] = mpi->planes[1];
        dmpi->stride[1] = mpi->stride[1];
        dmpi->planes[2] = mpi->planes[2];
        dmpi->stride[2] = mpi->stride[2];
    }

and pass dmpi along?

Second, as the example above illustrates, my filter will generally
output more Frames than it receives, and this has the potential of
messing up the a/v sync algorithm.  In fact, I *think* my prototype is
working, but the mencoder'ed result has very poor a/v sync (the video
lags significantly behind the audio, compared to the original).  I get
tons of "Skipping Frame!" and "1 duplicate frame(s)" messages, in
addition to messages from demux_mpg saying it's switching frame rates
between ~24Fps and ~30Fps.  Adding '-fps 30000/1001 -ofps 30000/1001'
makes the demux_mpg messages go away, but doesn't fix the a/v sync
problem.

What can/should I do to maintain a/v sync in this process?  I'm
clearing RFF of all Frames I pass down the chain, but that doesn't
seem to help.  Nor does following my filter with 'softskip'.  Is there
a way to have the mpeg2 decoder PROVIDE the RFF flag but not do the
normal timecode manipulation that it would ordinarily do upon
detecting it?  To wit, I see that 'pts' is now being passed through
the filter chain, but I'm not sure if I should pass it as-is, or
somehow track how my filter is affecting it and pass THAT.  Of if I
should just continue passing MP_NOPTS_VALUE.

Third, assuming I can make this work, does anyone have a suggestion
for what I call this thing? -- '-vf hardpulldown'?  Also, what file
should it go into? -- vf_softpulldown.c?  A new file?  I hacked it as
a new mode in vf_tinterlace.c for the time being, but that seems like
a very arbitrary and very poor choice in retrospect.

That's enough questions for now. :) Thanks for any help you folks can
provide.

Cheers,
WB


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com