[FFmpeg-devel] [PATCH] avcodec/h264: Fix segfault in 4:2:2 chroma deblock with 32-bit msvc

[James Darnley]

On 2016-02-05 21:20, Henrik Gramner wrote:
> Using rNm and x86inc's stack allocation with a negative value at the same
> time isn't supported, and caused the original stack pointer to be clobbered
> when using a compiler that doesn't support stack alignment.
> ---
>  libavcodec/x86/h264_deblock.asm | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/libavcodec/x86/h264_deblock.asm b/libavcodec/x86/h264_deblock.asm
> index 8f80863..4aabbc0 100644
> --- a/libavcodec/x86/h264_deblock.asm
> +++ b/libavcodec/x86/h264_deblock.asm
> @@ -867,7 +867,8 @@ ff_chroma_inter_body_mmxext:
>  %define t5 r4
>  %define t6 r5
>  
> -cglobal deblock_h_chroma422_8, 5, 6, 0, 0-(1+ARCH_X86_64*2)*mmsize
> +cglobal deblock_h_chroma422_8, 5, 6
> +    SUB rsp, (1+ARCH_X86_64*2)*mmsize
>      %if ARCH_X86_64
>          %define buf0 [rsp+16]
>          %define buf1 [rsp+8]
> @@ -907,6 +908,7 @@ cglobal deblock_h_chroma422_8, 5, 6, 0, 0-(1+ARCH_X86_64*2)*mmsize
>      movq m0, buf0
>      movq m3, buf1
>      TRANSPOSE8x4B_STORE PASS8ROWS(t5, r0, r1, t6)
> +    ADD rsp, (1+ARCH_X86_64*2)*mmsize
>  RET
>  
>  ; in: %1=p0 %2=p1 %3=q1
> 

Looks good to me but then I thought the old one did too so you probably
want to wait for someone else to sign off too.

Sorry everyone.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 603 bytes
Desc: OpenPGP digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20160205/6a48b1a0/attachment.sig>