[MPlayer-users] Fresh svn mplayer crashes on many ac3 streams

[Ivan Kalvachev]

On 5/25/11, Vladimir Mosgalin <mosgalin at vm10124.spb.edu> wrote:
> Hello everybody.
>
> I just compiled fresh mplayer svn
> (MPlayer SVN-r33497-4.6.0 (C) 2000-2011 MPlayer Team)
> with gcc-4.6.0-7 from Fedora 15. I experience crashes on lots of videos
> with ac3 streams. Some do play, some crash instantly at start of
> playback, some can crash randomly in the middle of file. It happens only
> with default ac3 decoder; -afm liba52 solves the problem and works
> flawlessly, however it's probably not a good thing that out-of-the-box
> mplayer crashes.
>
> What sucks is that mplayer with --enable-debug doesn't exhibit the
> crash. After some tracking down I found that the problem is -O4
> optimization flag of "normal" version vs -O2 for --enable-debug. The
> file "ac3dec.c" produces crash if it was compiled with -O3 or higher,
> but works fine for -O2. Crash seems to be unrelated to ac3 downmixing and
> happens both with -channels 6 and -channels 2.
>
> What is a bit puzzling is that if I dump ac3 stream off file with
> -dumpaudio, then mplayer plays it without crashes..
> It might be something related to muxing, as it looks like pack of the
> files that were encoded/muxed into mkv by a sole person can all crash in
> exactly the same way, and other packs of files work.
>
> Here are example backtraces, compiled with "normal" mplayer (without
> --enable-debug as that option overrides CFLAGS), but with -O3 -g among
> CFLAGS. Full list of CFLAGS for this example was
>
> -Wall -Wno-switch -Wno-parentheses -Wpointer-arith -Wredundant-decls -O3
> -march=corei7-avx -pipe -ffast-math -funroll-loops -fomit-frame-pointer
> -fexcess-precision=fast -g
> (and when whole mplayer compiled with these but ac3dec.c with these but
> -O2 instead of -O3, it works. -ffast-math -fexcess-precision=fast flags
> can be removed, it doesn't change anything - only some magic
> optimization from -O3 seems to produce this effect)
>
>
> Backtrace from video with ac3 stream that crashes right away:
>
> Program received signal SIGSEGV, Segmentation fault.
> set_downmix_coeffs (s=0x1f90850) at ac3dec.c:346
> 346         s->downmix_coeffs[i][1] *= norm1;
> (gdb) bt

This is amazing because [i][0] in the above line seems to be OK.

The -O3/2 is clear sign of misscompilation. It may not be gcc bug, if
the code in question is having some issue e.g. aliasing. It should
produce warning in that case.
(It would be useful the list of warning only for the ac3dec.c file)

However the code looks quite straight forward, without any hacks or abuses.


Anyway, If you are eager for a further debugging you can try to following ideas:

1. Test the cflag option -fno-tree-vectorize . FFmpeg have it by default.
Other possibly interesting flags -mno-sse -mno-mmx -mno-avx (if there
is such option).
Try to find if changing the -march to a older model fixes it at some point.
(i7-avx is quite cutting edge :)

2. valgrind. If the segfault is result of earlier memory corruption
this is the program that is likely to find it out. (avx may not be
supported):
Recommended options:

--log-file=error.log --undef-value-errors=no

(the last option is to silence many xlib/xcb related warnings that
always pop-up).


3. Compile with -O3. Change the config.mak to use -O2.
`touch` specific .c files, and rebuild with just `make` (from main
mplayer directory). Then try if it breaks.

In theory, if only one file is causing the crash, then if that file is
compiled with -O2 it would not crash, even if all other files are
compiled with -O3.

Good candidates for such files are ac3dec.c (from the error), and the
other ac3* in the mplayer/ffmpeg/libavcodec directory.

If this doesn't help, you can go brute force. To save some time. Do
the `touch` in bulk on whole directories. Aka sub-systems.
When you find the subsystem that fixes it, try to narrow down the files in it.