[FFmpeg-devel] [PATCH] lavc/qsv: improve the default GPU memory usage

[Zhong Li]

> A large initial_pool_size leads to redundant GPU memory allocations
> compared with MSDK.
>
> For some special cases which needs larger GPU memory like look_ahead,
> add -extra_hw_frames to allocate more.
>
> CMD:
> ffmpeg -hwaccel qsv -extra_hw_frames 50 -c:v hevc_qsv -i hevc.h265 -c:v h264_qsv
> -look_ahead 1 -look_ahead_depth 50 -y output.h264
>
> Fix #7943.
>
> Reported-by: Eero T
> Signed-off-by: Linjie Fu <linjie.fu at intel.com>
> ---
>  fftools/ffmpeg_qsv.c     | 2 +-
>  libavcodec/qsvdec.c      | 5 ++++-
>  libavcodec/qsvenc_h264.c | 4 ++--
>  3 files changed, 7 insertions(+), 4 deletions(-)
>
> diff --git a/fftools/ffmpeg_qsv.c b/fftools/ffmpeg_qsv.c
> index 9c4285b..5039a5a 100644
> --- a/fftools/ffmpeg_qsv.c
> +++ b/fftools/ffmpeg_qsv.c
> @@ -93,7 +93,7 @@ int qsv_init(AVCodecContext *s)
>      frames_ctx->height            = FFALIGN(s->coded_height, 32);
>      frames_ctx->format            = AV_PIX_FMT_QSV;
>      frames_ctx->sw_format         = s->sw_pix_fmt;
> -    frames_ctx->initial_pool_size = 64 + s->extra_hw_frames;
> +    frames_ctx->initial_pool_size = 24 + s->extra_hw_frames;

Should be better add pool size when look_ahead is enabled.

E:g
initial_pool_size = 24 + look_head_depth + s->extra_hw_frames;

you can set a default look_ahead_depth when look_ahead is enabled bu
no depth specified.

>      frames_hwctx->frame_type      = MFX_MEMTYPE_VIDEO_MEMORY_DECODER_TARGET;
>
>      ret = av_hwframe_ctx_init(ist->hw_frames_ctx);
> diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c
> index 0d34021..407e6de 100644
> --- a/libavcodec/qsvdec.c
> +++ b/libavcodec/qsvdec.c
> @@ -300,8 +300,11 @@ static int alloc_frame(AVCodecContext *avctx, QSVContext *q, QSVFrame *frame)
>      else
>          ret = ff_get_buffer(avctx, frame->frame, AV_GET_BUFFER_FLAG_REF);
>
> -    if (ret < 0)
> +    if (ret < 0) {
> +        av_log(avctx, AV_LOG_ERROR, "More memory in need, "
> +            "try -extra_hw_frames to allocate more.\n");
>          return ret;
> +    }
>
>      if (frame->frame->format == AV_PIX_FMT_QSV) {
>          frame->surface = *(mfxFrameSurface1*)frame->frame->data[3];
> diff --git a/libavcodec/qsvenc_h264.c b/libavcodec/qsvenc_h264.c
> index 27f36b9..4e8035b 100644
> --- a/libavcodec/qsvenc_h264.c
> +++ b/libavcodec/qsvenc_h264.c
> @@ -112,8 +112,8 @@ static const AVOption options[] = {
>      { "max_dec_frame_buffering", "Maximum number of frames buffered in the DPB", OFFSET(qsv.max_dec_frame_buffering), AV_OPT_TYPE_INT, { .i64 = 0 },   0, UINT16_MAX, VE },
>
>  #if QSV_HAVE_LA
> -    { "look_ahead",       "Use VBR algorithm with look ahead",    OFFSET(qsv.look_ahead),       AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE },
> -    { "look_ahead_depth", "Depth of look ahead in number frames", OFFSET(qsv.look_ahead_depth), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 100, VE },
> +    { "look_ahead",       "Use VBR algorithm with look ahead, extra_hw_frames is needed",    OFFSET(qsv.look_ahead),       AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 1, VE },
> +    { "look_ahead_depth", "Depth of look ahead in number frames, extra_hw_frames is needed", OFFSET(qsv.look_ahead_depth), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 100, VE },

No need to update the message if follow comments above.