[FFmpeg-devel] [PATCH v2 2/2] tests/fate: disable compression for zlib-based codecs

[James Almer]

On 10/16/2024 4:34 PM, Ramiro Polla wrote:
> On Mon, Oct 14, 2024 at 10:57 PM Michael Niedermayer
> <michael at niedermayer.cc> wrote:
>> On Sun, Sep 29, 2024 at 08:36:29PM +0200, Ramiro Polla wrote:
>>> FATE results differ when using the original zlib and zlib-ng.
>>>
>>> Since we don't need to test the result from zlib itself, this commit
>>> disables compression on tests for zlib-based codecs, which ends up
>>> giving the same results with both libraries.
>>> ---
>>>   tests/fate/cover-art.mak                  |  6 +--
>>>   tests/fate/image.mak                      |  4 +-
>>>   tests/fate/lavf-image.mak                 |  5 ++-
>>>   tests/fate/lavf-video.mak                 |  4 +-
>>>   tests/fate/mov.mak                        |  2 +-
>>>   tests/fate/vcodec.mak                     |  4 +-
>>>   tests/ref/fate/copy-apng                  |  4 +-
>>>   tests/ref/fate/cover-art-aiff-id3v2-remux |  6 +--
>>>   tests/ref/fate/cover-art-flac-remux       | 48 +++++++++++++++++------
>>>   tests/ref/fate/cover-art-mp3-id3v2-remux  |  6 +--
>>>   tests/ref/fate/mov-cover-image            |  6 +--
>>>   tests/ref/fate/png-icc                    |  6 +--
>>>   tests/ref/fate/png-mdcv                   |  4 +-
>>>   tests/ref/lavf/apng                       |  4 +-
>>>   tests/ref/lavf/apng.png                   |  4 +-
>>>   tests/ref/lavf/gray16be.png               |  4 +-
>>>   tests/ref/lavf/png                        |  4 +-
>>>   tests/ref/lavf/rgb48be.png                |  4 +-
>>>   tests/ref/seek/vsynth_lena-flashsv        | 40 +++++++++----------
>>>   tests/ref/vsynth/vsynth1-flashsv          |  4 +-
>>>   tests/ref/vsynth/vsynth1-mpng             |  4 +-
>>>   tests/ref/vsynth/vsynth1-zlib             |  4 +-
>>>   tests/ref/vsynth/vsynth2-flashsv          |  4 +-
>>>   tests/ref/vsynth/vsynth2-mpng             |  4 +-
>>>   tests/ref/vsynth/vsynth2-zlib             |  4 +-
>>>   tests/ref/vsynth/vsynth3-flashsv          |  4 +-
>>>   tests/ref/vsynth/vsynth3-mpng             |  4 +-
>>>   tests/ref/vsynth/vsynth3-zlib             |  4 +-
>>>   tests/ref/vsynth/vsynth_lena-flashsv      |  4 +-
>>>   tests/ref/vsynth/vsynth_lena-mpng         |  4 +-
>>>   tests/ref/vsynth/vsynth_lena-zlib         |  4 +-
>>>   31 files changed, 120 insertions(+), 93 deletions(-)
>>
>> with latest git i get this:
>>
>> make -j32 -k fate-png-mdcv fate-mov-cover-image
>> TEST    png-mdcv
>> TEST    mov-cover-image
>> --- ./tests/ref/fate/mov-cover-image    2024-10-14 22:49:44.488637985 +0200
>> +++ tests/data/fate/mov-cover-image     2024-10-14 22:56:27.175855259 +0200
>> @@ -1,4 +1,4 @@
>> -ecc0c7b60d4b00a2f04367c2ecd402c2 *tests/data/fate/mov-cover-image.mp4
>> +ad553810e0ce362697a2b21544daebde *tests/data/fate/mov-cover-image.mp4
>>   2063262 tests/data/fate/mov-cover-image.mp4
>>   #extradata 0:        2, 0x00340022
>>   #tb 0: 1/44100
>> @@ -20,7 +20,7 @@
>>   0,      -1088,      -1088,     1024,        6, 0x027e00e8, F=0x5
>>   0,        -64,        -64,     1024,        6, 0x027e00e8
>>   1,          0,          0,        0,    25441, 0xe82503b0
>> -2,          0,          0,        0,  1084000, 0x70fc8139
>> +2,          0,          0,        0,  1084000, 0x413c7ee9
>>   0,        960,        960,     1024,        6, 0x027e00e8
>>   0,       1984,       1984,     1024,        6, 0x027e00e8
>>   0,       3008,       3008,     1024,        6, 0x027e00e8
>> Test mov-cover-image failed. Look at tests/data/fate/mov-cover-image.err for details.
>> make: *** [tests/Makefile:311: fate-mov-cover-image] Error 1
>> --- ./tests/ref/fate/png-mdcv   2024-10-14 22:49:44.488637985 +0200
>> +++ tests/data/fate/png-mdcv    2024-10-14 22:56:27.227855667 +0200
>> @@ -1,5 +1,5 @@
>> -4001769a41d37d567c4cc0c532e20b54 *tests/data/fate/png-mdcv.image2pipe
>> -2774225 tests/data/fate/png-mdcv.image2pipe
>> +b674209fd9cd8eff945a6bc1a4b306d3 *tests/data/fate/png-mdcv.image2pipe
>> +2774240 tests/data/fate/png-mdcv.image2pipe
>>   #tb 0: 1/25
>>   #media_type 0: video
>>   #codec_id 0: rawvideo
>> Test png-mdcv failed. Look at tests/data/fate/png-mdcv.err for details.
>> make: *** [tests/Makefile:311: fate-png-mdcv] Error 1
> 
> The difference in the png tests comes from a bugfix in zlib itself:
> https://github.com/madler/zlib/commit/8ba393e70d984d902b15b9e6876f4d0d38ae4be8
> 
> The output of uncompressed data _may_ change between zlib versions
> 1.2.11 and 1.2.12.
> 
> We could just update the fate boxes to use zlib >= 1.2.12. Or write
> our own ff_deflate_uncompressed() which bypasses zlib. I can't think
> of anything better off the top of my head. Other suggestions are
> welcome.

The obvious solution is writing a ff_deflate_uncompressed() we would use 
for compression_level 0. Another is shipping zlib in our tree and adding 
a configure option to use the system one. But until any of them happens, 
I'll revert this patch.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: OpenPGP digital signature
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20241016/121ab245/attachment.sig>