[FFmpeg-devel] [PATCH v2 2/2] tests/fate: disable compression for zlib-based codecs

[Ramiro Polla]

On Mon, Oct 14, 2024 at 10:57 PM Michael Niedermayer
<michael at niedermayer.cc> wrote:
> On Sun, Sep 29, 2024 at 08:36:29PM +0200, Ramiro Polla wrote:
> > FATE results differ when using the original zlib and zlib-ng.
> >
> > Since we don't need to test the result from zlib itself, this commit
> > disables compression on tests for zlib-based codecs, which ends up
> > giving the same results with both libraries.
> > ---
> >  tests/fate/cover-art.mak                  |  6 +--
> >  tests/fate/image.mak                      |  4 +-
> >  tests/fate/lavf-image.mak                 |  5 ++-
> >  tests/fate/lavf-video.mak                 |  4 +-
> >  tests/fate/mov.mak                        |  2 +-
> >  tests/fate/vcodec.mak                     |  4 +-
> >  tests/ref/fate/copy-apng                  |  4 +-
> >  tests/ref/fate/cover-art-aiff-id3v2-remux |  6 +--
> >  tests/ref/fate/cover-art-flac-remux       | 48 +++++++++++++++++------
> >  tests/ref/fate/cover-art-mp3-id3v2-remux  |  6 +--
> >  tests/ref/fate/mov-cover-image            |  6 +--
> >  tests/ref/fate/png-icc                    |  6 +--
> >  tests/ref/fate/png-mdcv                   |  4 +-
> >  tests/ref/lavf/apng                       |  4 +-
> >  tests/ref/lavf/apng.png                   |  4 +-
> >  tests/ref/lavf/gray16be.png               |  4 +-
> >  tests/ref/lavf/png                        |  4 +-
> >  tests/ref/lavf/rgb48be.png                |  4 +-
> >  tests/ref/seek/vsynth_lena-flashsv        | 40 +++++++++----------
> >  tests/ref/vsynth/vsynth1-flashsv          |  4 +-
> >  tests/ref/vsynth/vsynth1-mpng             |  4 +-
> >  tests/ref/vsynth/vsynth1-zlib             |  4 +-
> >  tests/ref/vsynth/vsynth2-flashsv          |  4 +-
> >  tests/ref/vsynth/vsynth2-mpng             |  4 +-
> >  tests/ref/vsynth/vsynth2-zlib             |  4 +-
> >  tests/ref/vsynth/vsynth3-flashsv          |  4 +-
> >  tests/ref/vsynth/vsynth3-mpng             |  4 +-
> >  tests/ref/vsynth/vsynth3-zlib             |  4 +-
> >  tests/ref/vsynth/vsynth_lena-flashsv      |  4 +-
> >  tests/ref/vsynth/vsynth_lena-mpng         |  4 +-
> >  tests/ref/vsynth/vsynth_lena-zlib         |  4 +-
> >  31 files changed, 120 insertions(+), 93 deletions(-)
>
> with latest git i get this:
>
> make -j32 -k fate-png-mdcv fate-mov-cover-image
> TEST    png-mdcv
> TEST    mov-cover-image
> --- ./tests/ref/fate/mov-cover-image    2024-10-14 22:49:44.488637985 +0200
> +++ tests/data/fate/mov-cover-image     2024-10-14 22:56:27.175855259 +0200
> @@ -1,4 +1,4 @@
> -ecc0c7b60d4b00a2f04367c2ecd402c2 *tests/data/fate/mov-cover-image.mp4
> +ad553810e0ce362697a2b21544daebde *tests/data/fate/mov-cover-image.mp4
>  2063262 tests/data/fate/mov-cover-image.mp4
>  #extradata 0:        2, 0x00340022
>  #tb 0: 1/44100
> @@ -20,7 +20,7 @@
>  0,      -1088,      -1088,     1024,        6, 0x027e00e8, F=0x5
>  0,        -64,        -64,     1024,        6, 0x027e00e8
>  1,          0,          0,        0,    25441, 0xe82503b0
> -2,          0,          0,        0,  1084000, 0x70fc8139
> +2,          0,          0,        0,  1084000, 0x413c7ee9
>  0,        960,        960,     1024,        6, 0x027e00e8
>  0,       1984,       1984,     1024,        6, 0x027e00e8
>  0,       3008,       3008,     1024,        6, 0x027e00e8
> Test mov-cover-image failed. Look at tests/data/fate/mov-cover-image.err for details.
> make: *** [tests/Makefile:311: fate-mov-cover-image] Error 1
> --- ./tests/ref/fate/png-mdcv   2024-10-14 22:49:44.488637985 +0200
> +++ tests/data/fate/png-mdcv    2024-10-14 22:56:27.227855667 +0200
> @@ -1,5 +1,5 @@
> -4001769a41d37d567c4cc0c532e20b54 *tests/data/fate/png-mdcv.image2pipe
> -2774225 tests/data/fate/png-mdcv.image2pipe
> +b674209fd9cd8eff945a6bc1a4b306d3 *tests/data/fate/png-mdcv.image2pipe
> +2774240 tests/data/fate/png-mdcv.image2pipe
>  #tb 0: 1/25
>  #media_type 0: video
>  #codec_id 0: rawvideo
> Test png-mdcv failed. Look at tests/data/fate/png-mdcv.err for details.
> make: *** [tests/Makefile:311: fate-png-mdcv] Error 1

The difference in the png tests comes from a bugfix in zlib itself:
https://github.com/madler/zlib/commit/8ba393e70d984d902b15b9e6876f4d0d38ae4be8

The output of uncompressed data _may_ change between zlib versions
1.2.11 and 1.2.12.

We could just update the fate boxes to use zlib >= 1.2.12. Or write
our own ff_deflate_uncompressed() which bypasses zlib. I can't think
of anything better off the top of my head. Other suggestions are
welcome.