[FFmpeg-devel] [FFmpeg-cvslog] lavc/qsvenc: add support for external bitrate control for HEVC

[Fu, Linjie]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> Mark Thompson
> Sent: Thursday, February 20, 2020 07:00
> To: ffmpeg-devel at ffmpeg.org
> Subject: Re: [FFmpeg-devel] [FFmpeg-cvslog] lavc/qsvenc: add support for
> external bitrate control for HEVC
> 
> On 18/02/2020 02:32, Linjie Fu wrote:
> > ffmpeg | branch: master | Linjie Fu <linjie.fu at intel.com> | Thu Feb 13
> 20:47:24 2020 +0800| [ebee8085952de079946d903f0cc6e37aee3bc035] |
> committer: Zhong Li
> >
> > lavc/qsvenc: add support for external bitrate control for HEVC
> >
> > Enables option for hevc_qsv encoder:
> >     -extbrc
> >
> > Improvements in BD-Rate could be observed with extbrc on.
> >
> > Signed-off-by: Linjie Fu <linjie.fu at intel.com>
> > Signed-off-by: Zhong Li <zhongli_dev at 126.com>
> >
> >>
> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=ebee8085952de
> 079946d903f0cc6e37aee3bc035
> > ---
> >
> >  libavcodec/qsvenc.c | 18 ++++++++++++------
> >  1 file changed, 12 insertions(+), 6 deletions(-)
> >
> > diff --git a/libavcodec/qsvenc.c b/libavcodec/qsvenc.c
> > index 40904af388..52b4e43979 100644
> > --- a/libavcodec/qsvenc.c
> > +++ b/libavcodec/qsvenc.c
> > @@ -687,11 +687,8 @@ FF_ENABLE_DEPRECATION_WARNINGS
> >
> >          q->extparam_internal[q->nb_extparam_internal++] = (mfxExtBuffer
> *)&q->extco;
> >
> > -        if (avctx->codec_id == AV_CODEC_ID_H264) {
> >  #if QSV_HAVE_CO2
> > -            q->extco2.Header.BufferId     = MFX_EXTBUFF_CODING_OPTION2;
> > -            q->extco2.Header.BufferSz     = sizeof(q->extco2);
> > -
> > +        if (avctx->codec_id == AV_CODEC_ID_H264) {
> >              if (q->int_ref_type >= 0)
> >                  q->extco2.IntRefType = q->int_ref_type;
> >              if (q->int_ref_cycle_size >= 0)
> > @@ -703,8 +700,6 @@ FF_ENABLE_DEPRECATION_WARNINGS
> >                  q->extco2.BitrateLimit = q->bitrate_limit ?
> MFX_CODINGOPTION_ON : MFX_CODINGOPTION_OFF;
> >              if (q->mbbrc >= 0)
> >                  q->extco2.MBBRC = q->mbbrc ? MFX_CODINGOPTION_ON :
> MFX_CODINGOPTION_OFF;
> > -            if (q->extbrc >= 0)
> > -                q->extco2.ExtBRC = q->extbrc ? MFX_CODINGOPTION_ON :
> MFX_CODINGOPTION_OFF;
> >
> >              if (q->max_frame_size >= 0)
> >                  q->extco2.MaxFrameSize = q->max_frame_size;
> > @@ -752,9 +747,20 @@ FF_ENABLE_DEPRECATION_WARNINGS
> >                  q->extco2.MaxQPP = q->extco2.MaxQPB = q->extco2.MaxQPI;
> >              }
> >  #endif
> > +        }
> > +
> > +        if (avctx->codec_id == AV_CODEC_ID_H264 || avctx->codec_id ==
> AV_CODEC_ID_HEVC) {
> > +            if (q->extbrc >= 0)
> > +                q->extco2.ExtBRC = q->extbrc ? MFX_CODINGOPTION_ON :
> MFX_CODINGOPTION_OFF;
> > +
> > +            q->extco2.Header.BufferId = MFX_EXTBUFF_CODING_OPTION2;
> > +            q->extco2.Header.BufferSz = sizeof(q->extco2);
> > +
> >              q->extparam_internal[q->nb_extparam_internal++] =
> (mfxExtBuffer *)&q->extco2;
> > +        }
> >  #endif
> >
> > +        if (avctx->codec_id == AV_CODEC_ID_H264) {
> >  #if QSV_HAVE_MF
> >              if (QSV_RUNTIME_VERSION_ATLEAST(q->ver, 1, 25)) {
> >                  q->extmfp.Header.BufferId     =
> MFX_EXTBUFF_MULTI_FRAME_PARAM;
> 
> I am very confused by this.  Where is the implementation of the mfxExtBRC
> callbacks?  Who is actually doing the rate control which gives you this 20%
> better result?
> 

Sorry for being confusing, there are 2 types of external bitrate control available in MediaSDK:

1. Implicit External BRC, implemented inside MediaSDK lib [1];

2. Explicit External BRC, sample Implementation at application level[2] with callback functions;

This patch enables implicit External BRC and doing the rate control inside MediaSDK. 
In case of implicit (built-into-library) version - we don't need extended buffer for mfxextbrc.

[1] https://github.com/Intel-Media-SDK/MediaSDK/blob/master/samples/sample_encode/src/pipeline_encode.cpp#L639
[2] https://github.com/Intel-Media-SDK/MediaSDK/blob/master/doc/mediasdk-man.md#mfxextbrc

Thanks.