[FFmpeg-devel] [PATCH] configure: fix the bigendian test

[David Michael]

On Thu, Nov 12, 2020 at 2:56 AM Carl Eugen Hoyos <ceffmpeg at gmail.com> wrote:
> > Am 12.11.2020 um 00:41 schrieb David Michael <fedora.dm0 at gmail.com>:
> >> On Wed, Nov 11, 2020 at 6:20 PM Carl Eugen Hoyos <ceffmpeg at gmail.com> wrote:
> >>> Am Mi., 11. Nov. 2020 um 16:30 Uhr schrieb David Michael <fedora.dm0 at gmail.com>:
> >>> There are two issues:
> >>>
> >>> The unused global integer does not make it into the compiled object
> >>> file, so declare it static to correct this.
> >>
> >> How can I reproduce this issue?
> >
> > The configure test is basically these lines (where I use a PowerPC GCC
> > for a big-endian compiler):
> >
> > echo "unsigned int endian = 'B' << 24 | 'I' << 16 | 'G' << 8 | 'E';" |
> > powerpc-gentoo-linux-gnu-gcc $CFLAGS -c -o test.o -xc - &&
> > od -A n -t x1 test.o | tr -d '\n' | grep -o '42 *49 *47 *45'
> >
> > In testing this just now, it seems that the static value is only
> > present when "-O2 -g" is in CFLAGS, which I was always using as the
> > bare minimum flags from various distro packaging environments.  So
> > without static, the test is broken with "-flto"; with static, the test
> > is broken without "-O2 -g".  Maybe it could just check for a
> > _BIG_ENDIAN preprocessor definition instead.
>
> Sorry for the misunderstanding:
> What do I have to do (on the command line) to reproduce a problem with current FFmpeg git head?

Call the configure script to cross-compile to a big-endian
architecture with CFLAGS in the environment.  Basically run this with
your own compiler prefix:

./configure --enable-cross-compile --host-cc=gcc --arch=powerpc
--target-os=linux --cross-prefix=powerpc-unknown-linux-gnu-

If CFLAGS contains -flto, you'll see "big-endian no" in the output.
That results in compilation failures later due to missing PPC
definitions that were conditional on HAVE_BIGENDIAN.

Thanks.

David