[MPlayer-dev-eng] New inverse-telecine filter

[Zoltan Hidvegi]

This is a new inverse telecine filter, similar to detc, ivtc and pullup,
but better.  Here is a summary how it works.

The filter tries to detect frame boundaries by comparing fields.  Only
the luma plane is used for this comparison, since the chroma field is
usually too noisy and has lower resolution.  The filter works on
even-first (a.k.a. top-field first) frames, for a bottom-first frame the
first line is skipped to make it top-first.

Similar to the other filters, first a metric is calculated for each 8x8
pixel block not adjacent to the edge of the image.  Just like in the
other filters, the four components of the metrics are even, odd, noise,
temp.  Even and odd are the SAD (sum of absolute difference) between the
current field, and the same field in the previous frame.  Noise is the
difference between the two fields within the same frame.  It measures the
distance of the 4 odd lines in the 8x8 block from the even lines.  First
the even lines are used to approximate the image half a pixel below the
even lines, and the odd lines are used to approximate the image half a
pixel above the off lines, and for each pixel the difference between
these two are calculated.  Then the difference for half a pixel below the
odd lines is calculated, and the smaller of these two are kept for each
pixel.  This process eliminates false noise from sharp horizontal edges.
This difference is then capped by the difference between the current and
previous odd pixel, to reduce noise in 3-field long odd-even-odd frames.
The resulting 32 differences are added together to obtain the noise
metric of the block.

The temp metric is calculated similarly, but the odd (last) field from
the previous frame is compared to the even (first) field from the current
frame.

For each frame, a statistics of the block metrics is calculated in the
frame_stats structure.  For each metrics components there is a threshold
stored in the thres member of vf_priv_s.  In frame_starts, the tiny, low
and high members count the number of blocks above the threshold, twice the
threshold and 4 times the threshold respectively (each of these counts
have 4 components for the 4 components of the metrics).  The bigger
member counts the number of blocks where a metric component is 1.5 times
bigger and at least a half threshold bigger than its pair (pairs are
noise-temp, even-odd).  The twox member is a count of blocks where a
component is at least twice as much and at least threshold bigger than
its pair.  max is the maximum of all metrics (the original detc filter
used only this metric, but it is not used in this filter).

The sum of all metrics is also calculated in sad, but only for metrics
above the sad_thres, to avoid accumulating noise.

The interlaced_low member counts the number of blocks where the noise and
temp metrics are close to each other (neither is 1.5 times bigger than the
other), and both are above the threshold.  The interlaced_high count is
similar, but the metrics must be above twice the threshold.

To allow correct de-telecine of pulldown TV content with interlaced
overlay, the interlaced blocks (those counted in interlaced_low) are not
counted in the other statistics.

Using these statistics, the find_breaks routine tries to find frame
breaks, see the code for details.  This routine is probably not perfect
yet.

The above methods can detect frame boundaries quite reliably, but they
fail on still motionless frames.  When there is no clear motion, the
filter falls back using the telecine pattern, or the ratio of
input/output frames to mark the frames.  put_image will use the breaks
detected by find_breaks to output the frames, however, it will also
keep a strict input/output frame ratio, dropping or duplicating frames
as needed.  If frame timestamps were available, the filter could be
easily modified to use those instead of frame counting.  The
duplication does not mean that it actually outputs two frames for a
single input frame, as that is not really supported by mplayer-g1, but
it may show a 3-field long frame twice.  For hard-interlaces NTSC
content it will do 3-2 field skipping and de-interlacing with linear
approximation, which may cause some noticeable vertical flicker, but
it's better than nothing.  For 30 fps progressive content within a 24
fps movie, such as some TV ads, every 5th frame will be dropped.

This filter is safe to use on live TV input, it should keep A-V sync as
long as the correct -fps and -ofps options are used.  It is also quite
fast.  On Pentium III or better it uses MMX2, and the speed is limited by
the memory bandwidth more than by the CPU speed.  It has prefetch code
for 64-byte cache lines, this could be improved for 32-byte lines, but I
do not have time for that now.  At least the prefetch does not hurt the
performance on my Pentium III laptop, but improves it on my Athlon XP.
For non-MMX machines I use C mmx which uses packed C arithmetics with
unsigned long.  This is probably faster than byte-by-byte calculations,
but I really did this because I wrote everything with MMX2 first, and it
was easier to port it to C this way.

It has built-in crop support.  This is necessary for two reasons: the
external crop filter does not pass on the field flags, thus cannot be
used for soft-telecined content.  The external crop filter also messes up
the chroma interlacing on YV12 content when cropping non-multiple of 4
lines, which the built-in crop allows any even cropping.  The use of
external crop also prevents the use of direct rendering into the filter
buffers, thus the internal crop is faster.

See the manual page for the full list of options.  Only the English
manpage is updated, the Hungarian manpage documents none of the pullup
filters, this is no exception, and Hungarians use PAL anyways, and this
is mostly for NTSC.

Attached is a patch and a tar.gz containing the new files.

Zoli
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: pullup2.diff
URL: <http://lists.mplayerhq.hu/pipermail/mplayer-dev-eng/attachments/20031202/fae6ffcb/attachment.txt>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pullup2.tar.gz
Type: application/octet-stream
Size: 11038 bytes
Desc: 
URL: <http://lists.mplayerhq.hu/pipermail/mplayer-dev-eng/attachments/20031202/fae6ffcb/attachment.obj>