[FFmpeg-devel] [PATCH v4 2/2][GSoC 2024] tests/checkasm: Add check_vvc_sad to vvc_mc.c

[Rémi Denis-Courmont]

Le 21 mai 2024 09:37:18 GMT+03:00, "Martin Storsjö" <martin at martin.st> a écrit :
>On Tue, 21 May 2024, Rémi Denis-Courmont wrote:
>
>> Hi,
>> 
>> Le 20 mai 2024 03:42:03 GMT+03:00, Stone Chen <chen.stonechen at gmail.com> a écrit :
>>> Adds checkasm for DMVR SAD AVX2 implementation.
>>> 
>>> Benchmarks ( AMD 7940HS )
>>> vvc_sad_8x8_c: 70.0
>>> vvc_sad_8x8_avx2: 10.0
>>> vvc_sad_16x16_c: 280.0
>>> vvc_sad_16x16_avx2: 20.0
>>> vvc_sad_32x32_c: 1020.0
>>> vvc_sad_32x32_avx2: 70.0
>>> vvc_sad_64x64_c: 3560.0
>>> vvc_sad_64x64_avx2: 270.0
>>> vvc_sad_128x128_c: 13760.0
>>> vvc_sad_128x128_avx2: 1070.0
>>> ---
>>> tests/checkasm/vvc_mc.c | 38 ++++++++++++++++++++++++++++++++++++++
>>> 1 file changed, 38 insertions(+)
>> 
>> VVC benchmarks have increased checksam runtime by at least an order of magnitude. It's become so prohibitively slow that I could not even get to the end.
>> 
>> This is not an acceptable situation and impedes non-VVC assembler work
>
>I don't quite understand; whenever benchmarking anything in checkasm, I would always run e.g. "checkasm --test=ac3dsp --bench=ac3_sum_square_bufferfly_float", limiting the total running of tests to a specific module, and only benchmarking a subset of the run functions. (The --bench parameter specifies a prefix; only functions matching that prefix gets benchmarked.)

Sure that's how you do it when you're working on a specific new optimisation. Now we're trying to compare 128-bit and 256-bit vectors for *all* existing functions to see which ones need to be reworked.

That used to work (in 30 minutes on K230, 5 minutes on Zen 2, IIRC). Now it's effectively broken and that's not acceptable'

>
>Without limiting the scope with a --test parameter, checkasm benchmarking has always been prohibitively slow for me - so I don't think there's anything new here?

As said, it seems to be literally an order of magnitude slower than before if not worse.

>That said I'm not familiar with the VVC tests in checkasm, perhaps they benchmark things excessively. But I don't see how that would impede work on other DSP functions in any way?

James also complained about the same thing before I.