[FFmpeg-devel] [PATCH] dca: add new fate tests based on the dcadec-samples test suite

[Hendrik Leppkes]

On Tue, Feb 2, 2016 at 10:13 AM, Hendrik Leppkes <h.leppkes at gmail.com> wrote:
>>>
>>> The decoder works in float by default, and all other float codecs use
>>> this kind of test.
>>
>> mp3 fate tests use f32 mode, for example. Besides, most of those decoders output
>> 16bit audio when not using floats.
>>
>>> Not sure how much precision there actually is in the dca float signal
>>> for lossy, but if someone thinks its worse it we could try to use
>>> tiny_psnr in float or s32 mode (it doesn't do s24)
>>
>> tiny_psnr doesn't seem to do s32 either, only f32. How hard would it be to add
>> it, or s24?
>
> If anything I think we should make it test f32 since thats the native
> output mode, or stick to the usual s16 as converted by swr.
> Using the bitexact fixed point code path for those samples to get s24
> out of the decoder would leave the float path entirely untested.
>
>>
>>> Would require more careful tuning of the test though, as the precision
>>> on different systems will definitely differ then.
>>
>> Changing the tests to make them use f32 seems to need a FUZZ of at least 7.
>
> mp3 uses a fuzz of 17, so its likely going to require higher fuzz
> values on different systems.
> Will only find out after seeing it fail on FATE on a bunch of systems though.
>
>>
>> I'd like foo86 to chime in and give his opinion in the matter, but in any case
>> this is not a blocker so if you think s16 is fine then go ahead.
>
> We can also go with f32, its not going to be "worse" than a s16 test,
> just need to be prepared to update the fuzz values in the first couple
> days.
>

I did some testing on the fuzz values, and 9 is the lowest that makes
everything pass on my system.
References generated with cpuflags 0 and x87 fpu, and tested against
various combinations of increasing flags to determine the fuzz.

So do we just "try" with 9, and increase it where it fails?

New patch coming after this mail.

- Hendrik