[MPlayer-dev-eng] More on inverse telecine!

[Billy Biggs]

D Richard Felker III (dalias at aerifal.cx):

> > I'm doing pulldown detection in tvtime now, so I'm back in the
> > scene.  When you come across the titles, shouldn't you maintain the
> > current pulldown pattern so long as the results are consistent?
> > That is, I know what I try and do is, once I've found the phase,
> > unless I get numbers that conflict with it, maintain the current
> > phase.
> 
> That's what detc does. However, I've become very doubtful as to
> whether this is a good idea... More later.

  Sigh, yeah, this is the problem with heuristics... ;-)

> Well there are several metrics I use.

  Thanks a ton for describing them.  This is really helpful.  The
metrics I'm using are mostly blurred difference, so rather than take
into account noise later, blur the input before trying to detect
differences across fields.

> For each 8x8 block, I compute:
> 
> 1) Sum abs(this frame's even lines - prev frame's even lines).
> 2) Sum abs(this frame's odd lines - prev frame's odd lines).
> 
> And several 'noise' figures, computed as the n=4 fourier cofficient of
> each column, i.e. inner product with (+1,-1,+1,-1,+1,-1,+1,-1).
> 
> 3) Noise of prev. frame.
> 4) Noise of current frame.
> 5) Noise when even field of current frame is merged with odd field of
>    prev frame.
> 
> Then, once I have all these stats, I take the max of each individually
> over all 8x8 blocks, and I take the max of the following quantities as
> well:

  This is pretty cool.

> One dead giveaway that a frame is part of the telecine pattern is when
> the max even-odd is massively smaller than max odd-even. The
> difference is often as significant as 80 vs. 3000!

  Yes.

  The difference in mine is that I keep a history of the last 5 frames,
enough for one full sequence, and then look for the minimum difference
in the top and bottom sequences, and match that against a perfect
sequence, and see how many match.

  What's neat about the history/matching is that you can also match
against different cut positions, and try and keep probabilities around.
More on this after some experiments though.

  I think that even though history stuff can get too complicated, as
long as you keep your history REALLY SMALL, things are pretty much ok.

> it would have a hell of a time. The situation with Serial Experiments
> Lain is really bad too -- you should try it for testing your code! :)
> But even with normal, "well behaved" telecine, it's commonplace for
> edits to have been made after the telecine process, and that means
> sometimes a scene change will come right after the first telecine
> frame.
> 
> This isn't so bad, since you can just drop the frame and go ahead with
> the scene change, but there's also another nasty possibility! The
> first frame after the scene change can be one of the interlaced
> frames, and you have no previous reference frame to compare it against
> to tell if it's interlaced!

  This is hilarious.  Exactly the problem I had with a recording I did
of part of an episode of 'Power Rangers'.  Checking for these pulldown
edits was why I wrote my 'offline' algorithm in reetpvr.  It's two-pass,
and tries to narrow down on where edits occur, and I hand-wrote
strategires for each possibility.

  Of course for realtime playback, I don't have that luxury.

  Still, I should maybe better describe my offline algorithm's handling
of cuts to you as it might be helpful.  Old code is in CVS at
www.sf.net/projects/reetpvr/

> Right now, the only case I find it to be missing is that
> horrible one-line-per-frame vertical scrolling.

  I need to find an example of this, you're freaking me out. :)

> So, that brings us back to my original question... :(
> 
> By the way, one more thing about sticking to the telecine pattern.
> Lots of times, especially in anime, you have some motion, followed by
> a period of total stillness. If you track the pattern and keep merging
> frames during the stillness, you might actually reduce the picture
> quality, due to quantization errors in the source file/dvd and slight
> noise/discrepencies between the frames due to analog processing during
> production. This won't hurt visually in most cases, but it does waste
> bits if you'll be encoding.

  I think you'll get more pain from the NTSC chroma crawl artifacts.
Having a good filter for those will probably help a ton, and also help
tons for detecting the pulldown phase.

  But that said, the real thing missing here is something I was
specifically trying to avoid:  a metric for detecting 'interlaced
frames'.  I think that pulldown detection should be mostly biasd towards
the difference metrics, since they work best, but we should still have
some thresholds for 'should I bother', and only merge or whatever if
doing so would avoid comb artifacts.  Ugh ugh ugh though.

  -Billy