[FFmpeg-devel] x264_cqm_init segmentation fault with gcc 4.8

[Kelly Anderson]

I just updated my compiler the other day to gcc 4.8.0.  After rebuilding 
X264 i ran into a segmentation fault int x264_cqm_init and found the 
following post on the trac.

http://comments.gmane.org/gmane.comp.video.ffmpeg.trac/12875

No solution was detailed, so I did a bit of research into the latest gcc 
features to see what may have caused the problem.

As per:
http://gcc.gnu.org/gcc-4.8/changes.html

GCC now uses a more aggressive analysis to derive an upper bound for the 
number of iterations of loops using constraints imposed by language 
standards. This may cause non-conforming programs to no longer work as 
expected, such as SPEC CPU 2006 464.h264ref and 416.gamess. A new 
option, |-fno-aggressive-loop-optimizations|, was added to disable this 
aggressive analysis. In some loops that have known constant number of 
iterations, but undefined behavior is known to occur in the loop before 
reaching or during the last iteration, GCC will warn about the undefined 
behavior in the loop instead of deriving lower upper bound of the number 
of iterations for the loop. The warning can be disabled with 
|-Wno-aggressive-loop-optimizations|.

So I added -fno-aggressive-loop-optimizations to the flags and the 
segfault is gone.  If someone has some time maybe they could look at  
x264_cqm_init and see if they can tune it to work properly with 
aggressive loop optimizations turned on.

Regards,

Kelly Anderson