[FFmpeg-devel] [PATCH v6] gcc: Don't disable '-ftree-vectorize' if gcc version is 13 or newer

[Martin Storsjö]

On Mon, 16 Jun 2025, Martin Storsjö wrote:

> From: Jiawei <jiawei at iscas.ac.cn>
>
> This patch modifies the FFmpeg build system to allow GCC to use the
> `-ftree-vectorize` flag when the compiler version is 13 or newer.
> Enabling this flag can improve performance through better loop analysis
> and auto-vectorization (SIMD) opportunities in modern GCC versions.
>
> The explicit `-fno-tree-vectorize` flag was originally added in commit
> 973859f5230e (2009). A previous attempt to enable `-ftree-vectorize`
> was made in commit cb8646af24bd (2016), but it was reverted in
> fd6dbc53855f. The reason for the revert was that the inline x86 CABAC
> assembly code caused compiler errors - the compiler would run out of
> available registers during vectorization, making it unable to compile
> some functions. There were also reports of GCC hitting internal
> compiler errors, and miscompilations on x86_64.
>
> In commit 182663a58a7a (2023), the problematic CABAC function was made
> non-inline. This significantly reduces the risk of register exhaustion
> caused by inlining large assembly blocks, making the vectorizer safer
> to enable for other functions.
>
> Given improvements in GCC's vectorizer and the mitigation of the
> original issue, we now re-enable `-ftree-vectorize` for GCC version 13
> and above.
>
> Signed-off-by: Jiawei <jiawei at iscas.ac.cn>
> ---
> Reposting as iterations v3-v5 by Jiawei haven't made it to the
> mailing list.
>
> For future reference; the issues with the CABAC assembly is
> easily reproducible with current GCC versions too; build a version
> before 182663a58a7a (possibly backport
> effadce6c756247ea8bae32dc13bb3e6f464f0eb and
> f01fdedb69e4accb1d1555106d8f682ff1f1ddc7), configure a build
> for x86_32 with --cpu=haswell, and it hits errors like this:
>
> src/libavcodec/x86/cabac.h:199:5: error: ‘asm’ operand has impossible constraints
>
> The last time there were also reports of miscompilations,
> this were mentioned on the mailing list in
> https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2016-May/193915.html
> and
> https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2016-May/193977.html.
>
> If we apply this, let's hope that GCC has improved since last time;
> this is limited to be applied on only GCC 13 and newer (but for
> all architectures; in particular on aarch64 it is has been seen to
> be generally beneficial in e.g. dav1d).
>
> Also, note that this it not us explicitly opting in to an experimental
> feature, but this is a default feature in GCC that we've explicitly
> opted out from so far.
> ---
> configure | 6 +++++-
> 1 file changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/configure b/configure
> index 534b443f7d..117893ee93 100755
> --- a/configure
> +++ b/configure
> @@ -7673,7 +7673,11 @@ if enabled icc; then
>             disable aligned_stack
>     fi
> elif enabled gcc; then
> -    check_optflags -fno-tree-vectorize
> +    gcc_version=$($cc -dumpversion)
> +    major_version=${gcc_version%%.*}
> +    if [ $major_version -lt 13 ]; then
> +        check_optflags -fno-tree-vectorize
> +    fi
>     check_cflags -Werror=format-security
>     check_cflags -Werror=implicit-function-declaration
>     check_cflags -Werror=missing-prototypes
> -- 
> 2.39.5 (Apple Git-154)

The loongarch64 instance on patchwork does seem to be miscompiling 
something after this change, see 
https://patchwork.ffmpeg.org/project/ffmpeg/patch/20250616081107.57114-1-martin@martin.st/ 
and in particular https://patchwork.ffmpeg.org/check/125495/.

Not sure if the best way is to explicitly exclude known problematic 
architectures, or just do this for major architectures like x86 (maybe 
even exclude x86_32? or then again maybe the CABAC assembly was the only 
issue there), arm and aarch64 (maybe riscv)?

// Martin