[FFmpeg-devel] [PATCH] lavc/vaapi_encode: convert from lambda to qp

[Xiang, Haihao]

On Do, 2024-03-28 at 14:55 +0800, Xiang, Haihao wrote:
> From: Haihao Xiang <haihao.xiang at intel.com>
> 
> When AV_CODEC_FLAG_QSCALE is set, the value of avctx->global_quality is
> lambda.
> 
> Signed-off-by: Haihao Xiang <haihao.xiang at intel.com>
> ---
>  libavcodec/vaapi_encode.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
> index 940f0678a5..8b53095d61 100644
> --- a/libavcodec/vaapi_encode.c
> +++ b/libavcodec/vaapi_encode.c
> @@ -1961,7 +1961,10 @@ rc_mode_found:
>          if (ctx->explicit_qp) {
>              rc_quality = ctx->explicit_qp;
>          } else if (avctx->global_quality > 0) {
> -            rc_quality = avctx->global_quality;
> +            if (avctx->flags & AV_CODEC_FLAG_QSCALE)
> +                rc_quality = avctx->global_quality / FF_QP2LAMBDA;
> +            else
> +                rc_quality = avctx->global_quality;
>          } else {
>              rc_quality = ctx->codec->default_quality;
>              av_log(avctx, AV_LOG_WARNING, "No quality level set; "

Will apply,

- Haihao