[MPlayer-users] [PATCH] crash in libavcodec/i386/dsputil_mmx.c:float_to_int16_sse2

[Ray Kohler]

On Jul 11, 2008, at 1:10 PM, Reimar Döffinger wrote:

> Hello,
> On Thu, Jul 10, 2008 at 08:58:35PM -0400, Ray Kohler wrote:
>> The problem is that this code treats an unaligned doubleword as if it
>> were aligned (i.e., it segfaults when executing "movdqa").
>
> This is most likely a problem in MPlayer, FFmpeg requires the output
> buffer to be sufficiently aligned.

Found it. It's in ao_macosx. The address of the output buffer is  
ultimately derived from the value returned from get_space() in the AO  
module. ao_macosx faithfully reports the true amount of bits remaining  
in the ring buffer it allocates for CoreAudio to pull from. Since  
CoreAudio pulls samples rather than letting mplayer push them,  
ao_macosx cannot control (or even predict) whether this result is a  
multiple of 16. If it's not, the odd extra bytes will leave  
sh_audio_t.a_buffer_len not divisible by 16 after this round of audio  
decoding, causing the the output buffer to be misaligned on the next  
batch of decoding. (So I was wrong about it crashing on the first  
frame - it's always ok for at least one frame.)

My solution here is to make ao_macosx lie about the free space in the  
buffer by rounding down to a multiple of 16. As the function  
description says that the result wasn't guaranteed to be as large as  
the true value anyway, this shouldn't be a problem. In any case, my  
tests here have all passed.

Hopefully this change is more welcome than my last proposed patch. (I  
wasn't sure about commenting it or adding #ifdef HAVE_SSE2 and the  
like. Let me know if you want such.)

Index: libao2/ao_macosx.c
===================================================================
--- libao2/ao_macosx.c	(revision 27266)
+++ libao2/ao_macosx.c	(working copy)
@@ -115,7 +115,7 @@
  static int buf_free(void) {
    int free = ao->buf_read_pos - ao->buf_write_pos - ao->chunk_size;
    if (free < 0) free += ao->buffer_len;
-  return free;
+  return free - (free % 16);
  }

  /**