[FFmpeg-devel] [PATCH] checkasm/h264dsp: Fix stack overflow in check_idct_dequant

[Martin Storsjö]

On Mon, 16 Jun 2025, Zhao Zhili wrote:

> From: Zhao Zhili <zhilizhao at tencent.com>
>
> ---
> tests/checkasm/h264dsp.c | 14 ++++++++++----
> 1 file changed, 10 insertions(+), 4 deletions(-)
>
> diff --git a/tests/checkasm/h264dsp.c b/tests/checkasm/h264dsp.c
> index f5f9650224..006532e08b 100644
> --- a/tests/checkasm/h264dsp.c
> +++ b/tests/checkasm/h264dsp.c
> @@ -328,7 +328,7 @@ static void check_idct_multiple(void)
> static void check_idct_dequant(void)
> {
>     static const int depths[5] = { 8, 9, 10, 12, 14 };
> -    LOCAL_ALIGNED_16(int16_t, src, [16]);
> +    LOCAL_ALIGNED_16(int16_t, src, [16 * 2]);
>     /* Ensure dst buffers are large enough to hold dctcoefs of all bit-depths. */
>     LOCAL_ALIGNED_16(uint8_t, dst0, [16 * 16 * sizeof(int32_t)]);

Thanks for catching this! If the src buffer also is either int16_t or 
int32_t, can we make it uint8_t and use sizeof(int32_t) there as well just 
like dst, and move (and update) the comment above the src buffer too?

// Martin