[MPlayer-dev-eng] PostProcess SegFault fix 2

[Arpi]

Hi,

> I found this bug when testing -autoq. At maximum level
> on a mpeg1 video clip it segfaults after a while. The
> same happens on -pp 63 and -p32 so i was able to track
> the problem to chroma deringing in OpenDivX
> postprocess.c. 
> I am not the author of the fix, I just take a newer
> version of OpenDivX  and copy-paste, so this patch
> WILL NOT BREAK any LICENSE.
> P.S 
> I send this twice becouse the F. yahoo removed my
> attachment. I hope that now it won't :(
> 

Unfortunately this fix is wrong. :(

You've chanegd:
-//     for (y=8; y<height-8; y+=8) {
+       for (y=8; y<height-8; y+=8) {
(re-enabled the for loop).

it is bad, because the function is called with height==4.
this loop will never run -> this function does nothing.
it fixes the crashing this way :)

simple copypaste from opendivx code won't work, as i changed the structure
of this postprocessing code. the original (in opendivx releases/cvs) applies
the filters to the whole images. It has a very bad cache usage.
my version does 2 pass filtering, at first pass it applies all filters to
horizontal strides (8 pixel height) and the secodn pass it applies filters
to vertical strides. it gave me 2 times faster code.

this is why the for() loop here was commented out. the for is done outside
of the function, and it gets already incremented pointers.

i'll re-check this code, why can it crash.


A'rpi / Astral & ESP-team

--
mailto:arpi at thot.banki.hu
http://esp-team.scene.hu