[FFmpeg-devel] [PATCH v3] libavfilter: add a gblur_vulkan filter
Wu Jianhua
toqsxw at outlook.com
Sat Nov 6 11:05:49 EET 2021
Lynne wrote:
9 Sept 2021, 07:44 by jianhua.wu at intel.com:
> This commit adds a powerful and customizable gblur Vulkan filter,
> which provides a maximum 127x127 kernel size of Gaussian Filter.
> The size could be adjusted by requirements on quality or performance.
>
> The following command is on how to apply gblur_vulkan filter:
>
> ffmpeg -init_hw_device vulkan=vul:0 -filter_hw_device vul -i input.264
> -vf hwupload=extra_hw_frames=16,gblur_vulkan,hwdownload,format=yuv420p
> output.264
>
> Signed-off-by: Wu Jianhua <jianhua.wu at intel.com>
> ---
> configure | 1 +
> libavfilter/Makefile | 1 +
> libavfilter/allfilters.c | 1 +
> libavfilter/vf_gblur_vulkan.c | 511 ++++++++++++++++++++++++++++++++++
> 4 files changed, 514 insertions(+)
> create mode 100644 libavfilter/vf_gblur_vulkan.c
>
> diff --git a/configure b/configure
> index af410a9d11..4b9a0d8e07 100755
> --- a/configure
> +++ b/configure
> @@ -3601,6 +3601,7 @@ freezedetect_filter_select="scene_sad"
> frei0r_filter_deps="frei0r libdl"
> frei0r_src_filter_deps="frei0r libdl"
> fspp_filter_deps="gpl"
> +gblur_vulkan_filter_deps="vulkan_lib libglslang"
> histeq_filter_deps="gpl"
> hqdn3d_filter_deps="gpl"
> interlace_filter_deps="gpl"
> diff --git a/libavfilter/Makefile b/libavfilter/Makefile
> index af957a5ac0..5f74e33599 100644
> --- a/libavfilter/Makefile
> +++ b/libavfilter/Makefile
> @@ -286,6 +286,7 @@ OBJS-$(CONFIG_FREEZEFRAMES_FILTER) += vf_freezeframes.o
> OBJS-$(CONFIG_FREI0R_FILTER) += vf_frei0r.o
> OBJS-$(CONFIG_FSPP_FILTER) += vf_fspp.o qp_table.o
> OBJS-$(CONFIG_GBLUR_FILTER) += vf_gblur.o
> +OBJS-$(CONFIG_GBLUR_VULKAN_FILTER) += vf_gblur_vulkan.o vulkan.o
> OBJS-$(CONFIG_GEQ_FILTER) += vf_geq.o
> OBJS-$(CONFIG_GRADFUN_FILTER) += vf_gradfun.o
> OBJS-$(CONFIG_GRAPHMONITOR_FILTER) += f_graphmonitor.o
> diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
> index 0c6b2347c8..b5576de3af 100644
> --- a/libavfilter/allfilters.c
> +++ b/libavfilter/allfilters.c
> @@ -271,6 +271,7 @@ extern const AVFilter ff_vf_freezeframes;
> extern const AVFilter ff_vf_frei0r;
> extern const AVFilter ff_vf_fspp;
> extern const AVFilter ff_vf_gblur;
> +extern const AVFilter ff_vf_gblur_vulkan;
> extern const AVFilter ff_vf_geq;
> extern const AVFilter ff_vf_gradfun;
> extern const AVFilter ff_vf_graphmonitor;
> diff --git a/libavfilter/vf_gblur_vulkan.c b/libavfilter/vf_gblur_vulkan.c
> new file mode 100644
> index 0000000000..5c072f8971
> --- /dev/null
> +++ b/libavfilter/vf_gblur_vulkan.c
> @@ -0,0 +1,511 @@
> +/*
> + * copyright (c) 2021 Wu Jianhua <jianhua.wu at intel.com>
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +
> +#include "libavutil/random_seed.h"
> +#include "libavutil/opt.h"
> +#include "vulkan.h"
> +#include "internal.h"
> +
> +#define CGS 32
> +#define GBLUR_MAX_KERNEL_SIZE 127
> +
> +typedef struct GBlurVulkanContext {
> + VulkanFilterContext vkctx;
> + FFVkExecContext *exec;
> + VulkanPipeline *pl_hor;
> + VulkanPipeline *pl_ver;
> + FFVkBuffer params_buf_hor;
> + FFVkBuffer params_buf_ver;
> +
> + VkDescriptorImageInfo input_images[3];
> + VkDescriptorImageInfo tmp_images[3];
> + VkDescriptorImageInfo output_images[3];
> + VkDescriptorBufferInfo params_desc_hor;
> + VkDescriptorBufferInfo params_desc_ver;
> +
> + int initialized;
> + int size;
> + int planes;
> + int kernel_size;
> + float sigma;
> + float sigmaV;
> + AVFrame *tmpframe;
> +} GBlurVulkanContext;
> +
> +static const char gblur_horizontal[] = {
> + C(0, void gblur(const ivec2 pos, const int index) )
> + C(0, { )
> + C(1, vec4 sum = texture(input_image[index], pos) * kernel[0]; )
> + C(0, )
> + C(1, for(int i = 1; i < kernel.length(); i++) { )
> + C(2, sum += texture(input_image[index], pos + vec2(i, 0.0)) * kernel[i]; )
> + C(2, sum += texture(input_image[index], pos - vec2(i, 0.0)) * kernel[i]; )
> + C(1, } )
> + C(0, )
> + C(1, imageStore(output_image[index], pos, sum); )
> + C(0, } )
> +};
> +
> +static const char gblur_vertical[] = {
> + C(0, void gblur(const ivec2 pos, const int index) )
> + C(0, { )
> + C(1, vec4 sum = texture(input_image[index], pos) * kernel[0]; )
> + C(0, )
> + C(1, for(int i = 1; i < kernel.length(); i++) { )
> + C(2, sum += texture(input_image[index], pos + vec2(0.0, i)) * kernel[i]; )
> + C(2, sum += texture(input_image[index], pos - vec2(0.0, i)) * kernel[i]; )
> + C(1, } )
> + C(0, )
> + C(1, imageStore(output_image[index], pos, sum); )
> + C(0, } )
> +};
>
> The reason why avgblur_vulkan is split into horizontal and vertical
> was because you can change the blur radius in either direction.
> This is always going to be square, so to speed it up significantly,
> you can just do it all at once in both directions.
Hi Lynne,
Looks like I have the same functionality here. The sigma for horizontal
and the sigmaV for vertical could be different. If the sigmaV is not set,
then it is simply a 1 dimension horizontal Gaussian Blur. It might be a
flexible way for users. And about the performance, we have enough
tests that it's still on the expectation.
Thanks,
Jianhua
> By the way, I've written a replacement for the synchronization
> system that uses timeline semaphores. The repo is at
> https://github.com/cyanreg/FFmpeg/tree/vulkan
> I'll be pushing it in a few days once someone reviews it.
> You can adapt your patches to that.
> I've CC'd Wenbin Chen here too. I think your single_memory
> flag patch is reasonable, could you rebase it onto the branch and
> resubmit?
>
More information about the ffmpeg-devel
mailing list