[FFmpeg-devel] [PATCH] fix h264_deblock_sse2.asm segfaults on clang/x86-32

[Ronald S. Bultje]

Hi,

On Tue, Aug 31, 2010 at 7:06 PM, Ronald S. Bultje <rsbultje at gmail.com> wrote:
> Attached patch tries to make it work with any alignment. x86-64
> h_luma_intra wasn't fixed because it doesn't show any obvious issues
> and any attempt of mine to fix it bombed out, probably somewhere in
> the macros which do way much. This code is scary witchcraft, almost as
> crazy as inline asm. With this patch, most sigbus issues are gone and
> "make fate-h264" goes well until about halfway

Attached is a new patch with all x86-64 parts removed (requested by
Mans, since they don't crash on fate), so this only touches x86-32
specific code. Effect on runtime should be minimal, but b/c of the
stack saving this will be 1-2 instructions more for each function.

> (h264-conformance-caba2_sony_e), where it crashes somewhere in
> decode_cabac_mb_mvd() which I haven't quite traced down yet.

Turns out to be a compiler bug, see http://llvm.org/bugs/show_bug.cgi?id=8044

Ronald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fix-clang-h264-alignment.patch
Type: application/octet-stream
Size: 5284 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20100831/69eb3c1e/attachment.obj>