[FFmpeg-devel] [PATCH] correct make test failure from 15261 release until now (15899)

[Måns Rullgård]

David Geldreich <david.geldreich at free.fr> writes:

> Hi all,
>
> Le 21 nov. 08 ? 22:13, Siarhei Siamashka a ?crit :
>
>> On Friday 21 November 2008, Michael Niedermayer wrote:
>>> On Fri, Nov 21, 2008 at 09:15:30PM +0100, Vitor Sessak wrote:
>>>> Even if we find a way to generate a sine window in an arch- 
>>>> independent
>>>> way, the codec still uses floating points in other places, so if  
>>>> it ever
>>>> is bit-identical across PPC and I32, I don't see any reason not to  
>>>> see a
>>>> different output when testing on ARM or SH or GCC 6.4 or whatever  
>>>> we'll
>>>> encounter in future. Unless someone tells me why it is supposed to  
>>>> work
>>>> as is, I think that this test should be removed...
>>>
>>> ratecontrol in video uses floats, and other parts do too, we arent  
>>> seeing
>>> problems with these and arent disabling them
>>>
>>> If you argue that the wma test should be disabled because it is not
>>> matching between some important systems, thats something i can  
>>> understand
>>>
>>> but, arguing that itz should be disabled because it might  
>>> theoretically not
>>> work on some architecture or not yet existing compiler is well ...
>>>
>>> And last, id say disable the float code for the regression tests,
>>> replacing the whole fft by a memcpy() if it doesnt match beteen archs
>>> is alot better than removing a regression test.
>>> These tests are important to catch bugs early ...
>
> I think there is multiple option to do these float tests consistently :
>
> 1)  build the regression test with IEEE float support so float will  
> gives the same result on all platform with a compiler/libc/libm  
> supporting IEEE floats.

IEEE754 allows some variation.

> 2) from each file, extract the bytes that represent "headers" data  
> (and are not-float/stable across target) do a MD5 on it.
> extract the float part and do a Chi-square or SSD-style test using an  
> error threshold. For the data part, perhaps once encoded Chi-square is  
> not meaningful, in this case, decode the data to do the test.

The regression test much detect any change in output, no matter how
small.  What you suggest would be reasonable for a conformance test,
but not for a regression test.

> Solution 1) will be the easiest to implement but will not detect  
> problem with "normal" (not IEEE) build. But is this a real problem ?  
> today, a "make test" on a PPC Mac gives different results if I use -- 
> enable-altivec or --disable-altivec.

It is likely that altivec uses lower precision or different rounding
to speed up calculations.

-- 
M?ns Rullg?rd
mans at mansr.com