[FFmpeg-devel] [PATCH] lavfi: add colorkey_opencl filter

[Cld fire]

On Wed, Apr 10, 2019 at 6:10 PM Mark Thompson <sw at jkqxz.net> wrote:

> The lack of checks in overlay_opencl generally leaves a lot to be
> desired.  It's is intended for (and only really tested with) hardware video
> in YUV formats (primarily overlaying YUV or YUVA on YUV, for things like
> subtitles or pips), but is rather variable in how well it works at that.
>
> A tiny bit of hacking can get me to something which works with RGBA here.
> Changing:
>
> diff --git a/libavfilter/vf_overlay_opencl.c
> b/libavfilter/vf_overlay_opencl.c
> index e9c853203b..09070e106a 100644
> --- a/libavfilter/vf_overlay_opencl.c
> +++ b/libavfilter/vf_overlay_opencl.c
> @@ -81,7 +81,7 @@ static int overlay_opencl_load(AVFilterContext *avctx,
>      }
>
>      if (main_planes == overlay_planes) {
> -        if (main_desc->nb_components == overlay_desc->nb_components)
> +        if (main_desc->nb_components == overlay_desc->nb_components && 0)
>              kernel = "overlay_no_alpha";
>          else
>              kernel = "overlay_internal_alpha";
>
> to force the upper layer alpha to take effect directly I can make it work
> to blend one RGBA video on top of another with:
>
> ./ffmpeg_g -y -init_hw_device vaapi=va:/dev/dri/renderD128 -init_hw_device
> opencl=ocl at va -hwaccel vaapi -hwaccel_output_format vaapi -i in1.mp4
> -hwaccel vaapi -hwaccel_output_format vaapi -i in2.mp4 -an
> -filter_hw_device ocl -filter_complex
> '[0:v]scale_vaapi=format=rgba,hwmap,colorkey_opencl=white:0.4:0.0[over];[1:v]scale_vaapi=format=rgba,hwmap[under];[under][over]overlay_opencl,hwmap=derive_device=vaapi:reverse=1,scale_vaapi=format=nv12[out]'
> -map '[out]' -c:v h264_vaapi out.mp4
>
> (Intel GPU running Beignet for interop, the whole thing manages close to
> 100fps at 1080p.)
>
> Patches welcome to make overlay_opencl not be so flaky, I guess :)
>

Okay, thanks for looking into that for me. I'll spend some time
investigating it on my end over the next few days.

Since blend is a constant per filter invocation, is there any gain to
> making two separate kernels and removing the if-branch entirely?
>

I'll try that out.

> +        // Make sure the input is a format we support
> > +        if (fmt != AV_PIX_FMT_ARGB &&
> > +            fmt != AV_PIX_FMT_RGBA &&
> > +            fmt != AV_PIX_FMT_ABGR &&
> > +            fmt != AV_PIX_FMT_BGRA &&
> > +            fmt != AV_PIX_FMT_NONE
>
> Why NONE here?
>

It was in the list of formats that the CPU filter said it supported and,
after briefly looking around, it appeared
that it was used as a generic special case of some kind (?). There's no
documentation on the enum variant,
though, so wasn't able to figure out its exact meaning. Judging by your
question, perhaps it is irrelevant for
HW frames?