[FFmpeg-devel] [PATCH v2] configure: fix the bigendian test

[David Michael]

On Sat, Nov 14, 2020 at 6:57 PM Carl Eugen Hoyos <ceffmpeg at gmail.com> wrote:
> Am Sa., 14. Nov. 2020 um 20:16 Uhr schrieb David Michael <fedora.dm0 at gmail.com>:
> > On Sat, Nov 14, 2020 at 2:06 PM Carl Eugen Hoyos <ceffmpeg at gmail.com> wrote:
> > > Am Sa., 14. Nov. 2020 um 18:22 Uhr schrieb David Michael <fedora.dm0 at gmail.com>:
> > > >
> > > > There are two issues:
> > > >
> > > > When build systems enable LTO in CFLAGS, the unused global integer
> > > > does not make it into the compiled object file.  As a workaround,
> > > > check if the compiler understands -fno-lto and append it after
> > > > CFLAGS while building the endianness test.
> > > >
> > > > The hexdump output is line-wrapped, so the expected value will not
> > > > be matched when its four bytes are split across two lines.  Use the
> > > > POSIX "-A n" option to disable printing input offsets and delete
> > > > all newline characters to output continuous hex values to grep.
> > >
> > > Please explain how we can reproduce the issue (configure line,
> > > compiler, os).
> >
> > It's the same command I sent you for the v1 patch.  Just configure it
> > to cross-compile to a big-endian target with CFLAGS=-flto in the
> > environment:
> >
> > CFLAGS=-flto ./configure --enable-cross-compile --host-cc=gcc
> > --arch=powerpc --target-os=linux
> > --cross-prefix=powerpc-unknown-linux-gnu-
>
> Is the issue reproducible if you configure with "--enable-lto" instead
> of CFLAGS (I suspect FFmpeg does not support CFLAGS in general)?

There is no failure when it has empty CFLAGS and --enable-lto since
that adds -flto after the endianness test.  If it is expected that
ffmpeg will break from CFLAGS, then I guess I can pursue this at the
distro packaging level.

Thanks.

David