[FFmpeg-devel] [PATCH] Hack around gcc 4.6 breaking asm using call.

[Reimar Döffinger]

On Wed, Sep 21, 2011 at 06:27:35AM +0400, Yuriy Kaminskiy wrote:
> Reimar Döffinger wrote:
> > On Tue, Sep 20, 2011 at 10:59:13PM +0400, Yuriy Kaminskiy wrote:
> >> Reimar Döffinger wrote:
> >>> gcc 4.6 no longer decrements esp to account for local variables.
> >>> Thus using call will end up overwriting some local variable.
> >>> So add an extra one it can safely clobber.
> >>> This is a huge hack because it's basically pure chance it works,
> >>> no idea how this is supposed to be done.
> >>>
> >>> Fixes trac ticket #397.
> >>>
> >>> Signed-off-by: Reimar Döffinger <Reimar.Doeffinger at gmx.de>
> >>> ---
> >>>  libswscale/x86/swscale_template.c |   10 ++++++++++
> >>>  1 files changed, 10 insertions(+), 0 deletions(-)
> >>>
> >>> diff --git a/libswscale/x86/swscale_template.c b/libswscale/x86/swscale_template.c
> >>> index 6196e98..d65be48 100644
> >>> --- a/libswscale/x86/swscale_template.c
> >>> +++ b/libswscale/x86/swscale_template.c
> >>> @@ -2283,6 +2283,10 @@ static void RENAME(hyscale_fast)(SwsContext *c, int16_t *dst,
> >>>  #if defined(PIC)
> >>>      DECLARE_ALIGNED(8, uint64_t, ebxsave);
> >>>  #endif
> >>> +    // HACK: gcc 4.6 no longer decrements esp,
> >>> +    // use this to make it reserve space for the call
> >>> +    // return address
> >>> +    void *dummy;
> >>>  
> >>>      __asm__ volatile(
> >>>  #if defined(PIC)
> >>> @@ -2334,6 +2338,7 @@ static void RENAME(hyscale_fast)(SwsContext *c, int16_t *dst,
> >>>  #if defined(PIC)
> >>>            ,"m" (ebxsave)
> >>>  #endif
> >>> +          ,"m" (dummy)
> >> Hmm, I'm not gcc/assembler expert, but here ebxsave is *input* parameter, but
> >> uninitialized before, not marked as *output* parameter, and there are no
> >> "memory" in clobbers (but it is actually modified/used in asm). So I won't even
> >> consider this gcc bug (or even "surprising behavior"). And it won't surprise me
> >> if your workaround will be broken again by next gcc version.
> > 
> > Well, what then is the correct way to tell gcc "hey, I will use the call
> > instruction here, don't break it". Not being able to use call at all in
> > asm code can hardly be considered sensible?
> 
> Oops, looked at assembler output, above is unrelated, sorry for noise.
> But found WTF this breaks: google://"amd64 red-zone".
> I think compiling this file (and any other that have call inside asm) with
> -mno-red-zone option would be safer.

Sure that works? I was quite convinced that option only works for PPC
and is completely ignored on x86...
But I see that the search indeed seems to point to people for whom it
works...