[FFmpeg-devel] [PATCH v2 1/2] qsvdec: add support for HW_DEVICE_CTX method
Soft Works
softworkz at hotmail.com
Fri Jul 30 11:18:40 EEST 2021
> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> Haihao Xiang
> Sent: Thursday, 29 July 2021 09:04
> To: ffmpeg-devel at ffmpeg.org
> Cc: Haihao Xiang <haihao.xiang at intel.com>
> Subject: [FFmpeg-devel] [PATCH v2 1/2] qsvdec: add support for
> HW_DEVICE_CTX method
>
> This allows user set hw_device_ctx instead of hw_frames_ctx for QSV
> decoders, hence we may remove the ad-hoc libmfx setup code from
> FFmpeg.
>
> "-hwaccel_output_format format" is applied to QSV decoders after removing
> the ad-hoc libmfx code. To keep compatibility with old commandlines, the
> default format is set to AV_PIX_FMT_QSV. User should set "-
> hwaccel_output_format qsv" explicitly if AV_PIX_FMT_QSV is expected.
>
> User may use the normal way to set device for QSV decoders now.
> "-qsv_device device" is deprecated and will be removed from FFmpeg.
>
> For example:
>
> $> ffmpeg -init_hw_device vaapi=va:/dev/dri/card0 -init_hw_device
> qsv=hw at va -hwaccel qsv -c:v h264_qsv -i input.h264 -f null -
Hi Haihao,
Why so complicated with double device initialization?
The regular way is this:
ffmpeg -init_hw_device "qsv=dev:hw_any,child_device=/dev/dri/card0" -hwaccel qsv -c:v h264_qsv -i input.h264 -f null -
Or for Windows:
ffmpeg -init_hw_device "qsv=dev:hw_any,child_device=1" -hwaccel qsv -c:v h264_qsv -i input.h264 -f null -
>
> /dev/dri/renderD128 is actually open for h264_qsv decoder without this
> patch. After applying this patch, /dev/dri/card0 is used.
> ---
> v2: Rebase the patchset against the latest master branch and resolve a
> conflict.
>
> fftools/Makefile | 1 -
> fftools/ffmpeg.h | 1 -
> fftools/ffmpeg_hw.c | 12 +++++
> fftools/ffmpeg_opt.c | 54 +++++++++++++++++++-- fftools/ffmpeg_qsv.c
> | 109 -------------------------------------------
> libavcodec/qsvdec.c | 31 +++++++++++-
> 6 files changed, 92 insertions(+), 116 deletions(-) delete mode 100644
> fftools/ffmpeg_qsv.c
>
> diff --git a/fftools/Makefile b/fftools/Makefile index 5affaa3f56..5234932ab0
> 100644
> --- a/fftools/Makefile
> +++ b/fftools/Makefile
> @@ -10,7 +10,6 @@ ALLAVPROGS =
> $(AVBASENAMES:%=%$(PROGSSUF)$(EXESUF))
> ALLAVPROGS_G = $(AVBASENAMES:%=%$(PROGSSUF)_g$(EXESUF))
>
> OBJS-ffmpeg += fftools/ffmpeg_opt.o fftools/ffmpeg_filter.o
> fftools/ffmpeg_hw.o
> -OBJS-ffmpeg-$(CONFIG_LIBMFX) += fftools/ffmpeg_qsv.o
> ifndef CONFIG_VIDEOTOOLBOX
> OBJS-ffmpeg-$(CONFIG_VDA) += fftools/ffmpeg_videotoolbox.o
> endif
> diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h index
> d9c0628657..d2dd7ca092 100644
> --- a/fftools/ffmpeg.h
> +++ b/fftools/ffmpeg.h
> @@ -61,7 +61,6 @@ enum HWAccelID {
> HWACCEL_AUTO,
> HWACCEL_GENERIC,
> HWACCEL_VIDEOTOOLBOX,
> - HWACCEL_QSV,
> };
>
> typedef struct HWAccel {
> diff --git a/fftools/ffmpeg_hw.c b/fftools/ffmpeg_hw.c index
> fc4a5d31d6..043e4c5863 100644
> --- a/fftools/ffmpeg_hw.c
> +++ b/fftools/ffmpeg_hw.c
> @@ -339,6 +339,18 @@ int hw_device_setup_for_decode(InputStream *ist)
> } else if (ist->hwaccel_id == HWACCEL_GENERIC) {
> type = ist->hwaccel_device_type;
> dev = hw_device_get_by_type(type);
> +
> + // When "-qsv_device device" is used, an internal QSV device named
> + // as "__qsv_device" is created and another QSV device is created
> + // if "-init_hw_device qsv=name at name" is used. There are 2 QSV
> devices
> + // if both "-qsv_device device" and "-init_hw_device
> qsv=name at name"
> + // are used, hw_device_get_by_type(AV_HWDEVICE_TYPE_QSV)
> returns NULL.
> + // To keep back-compatibility with the removed ad-hoc libmfx setup
> code,
> + // call hw_device_get_by_name("__qsv_device") to select the
> internal QSV
> + // device. This will be removed once -qsv_device is no longer
> supported.
> + if (!dev && type == AV_HWDEVICE_TYPE_QSV)
> + dev = hw_device_get_by_name("__qsv_device");
> +
> if (!dev)
> err = hw_device_init_from_type(type, NULL, &dev);
> } else {
> diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c index
> 1b43bab9fc..4ddc6c939b 100644
> --- a/fftools/ffmpeg_opt.c
> +++ b/fftools/ffmpeg_opt.c
> @@ -137,9 +137,6 @@ static const char *const opt_name_enc_time_bases[]
> = {"enc_time_base"
> const HWAccel hwaccels[] = {
> #if CONFIG_VIDEOTOOLBOX
> { "videotoolbox", videotoolbox_init, HWACCEL_VIDEOTOOLBOX,
> AV_PIX_FMT_VIDEOTOOLBOX }, -#endif -#if CONFIG_LIBMFX
> - { "qsv", qsv_init, HWACCEL_QSV, AV_PIX_FMT_QSV },
> #endif
> { 0 },
> };
> @@ -571,6 +568,49 @@ static int opt_vaapi_device(void *optctx, const char
> *opt, const char *arg) } #endif
>
> +#if CONFIG_QSV
> +static int opt_qsv_device(void *optctx, const char *opt, const char
> +*arg) { #if CONFIG_VAAPI
> + const char *prefix = "vaapi=__qsv_child_device:"; #elif
> +CONFIG_DXVA2
> + const char *prefix = "dxva2=__qsv_child_device:"; #else
> + const char *prefix = NULL;
> +#endif
Same like for the command line. You can directly initialize a QSV context
with a single operation.
[...]
> + if (avctx->hw_device_ctx && !avctx->hw_frames_ctx && ret ==
> AV_PIX_FMT_QSV) {
> + AVHWFramesContext *hwframes_ctx;
> + AVQSVFramesContext *frames_hwctx;
> +
> + avctx->hw_frames_ctx =
> + av_hwframe_ctx_alloc(avctx->hw_device_ctx);
> +
> + if (!avctx->hw_frames_ctx) {
> + av_log(avctx, AV_LOG_ERROR, "av_hwframe_ctx_alloc failed\n");
> + return AVERROR(ENOMEM);
> + }
> +
> + hwframes_ctx = (AVHWFramesContext*)avctx->hw_frames_ctx-
> >data;
> + frames_hwctx = hwframes_ctx->hwctx;
> + hwframes_ctx->width = FFALIGN(avctx->coded_width, 32);
> + hwframes_ctx->height = FFALIGN(avctx->coded_height, 32);
> + hwframes_ctx->format = AV_PIX_FMT_QSV;
> + hwframes_ctx->sw_format = avctx->sw_pix_fmt;
> + hwframes_ctx->initial_pool_size = 64 + avctx->extra_hw_frames;
I'm not sure whether 64 is a good default choice. I had to reduce this to 32 when processing 4k video with OpenCL interop (memory sharing), otherwise I got OOM errors. (with video mem set to the maximum of 1G in the BIOS)
Besides the above comments, this looks good to me in general. It is the right step to go forward IMO.
Just the double device initialization should not be required (it isn't now) when it comes to the selection between D3D9 and D3D11. I am simply using a device parameter for this, and a command line looks like this:
ffmpeg -init_hw_device "qsv=dev:hw_any,child_device=1,dx11=1" -hwaccel qsv -c:v h264_qsv -i input.h264 -f null -
But let's get this done done first..
softworkz
More information about the ffmpeg-devel
mailing list