[FFmpeg-devel] [PATCH 1/2] avcodec/videotoolboxenc: add hw_configs

[Zhao Zhili]

> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of Andreas Rheinhardt
> Sent: 2023年10月12日 22:56
> To: ffmpeg-devel at ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH 1/2] avcodec/videotoolboxenc: add hw_configs
> 
> Zhao Zhili:
> >
> >> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of Andreas Rheinhardt
> >> Sent: 2023年10月12日 22:01
> >> To: ffmpeg-devel at ffmpeg.org
> >> Subject: Re: [FFmpeg-devel] [PATCH 1/2] avcodec/videotoolboxenc: add hw_configs
> >>
> >> Zhao Zhili:
> >>> From: Zhao Zhili <zhilizhao at tencent.com>
> >>>
> >>> Will be used in the following patches.
> >>
> >> It doesn't seem to be used for 2/2, so it should be applied with the
> >> patches that actually need it.
> >
> > It's used in an nonobvious way. With hw_config we can get avctx->hw_frames_ctx,
> > and with avctx->hw_frames_ctx we get sw_pix_fmt. Otherwise sw_pix_fmt is none.
> > I need sw_pix_fmt before get the first frame.
> >
> 
> So setting these hw_config magically sets AVCodecContext.sw_pix_fmt? How
> does this work? Does this work magically with callers other than the
> ffmpeg cli?

It's done by encode.c encode_preinit_video() to copy hw_frames_ctx sw_format to avctx->sw_pix_fmt.
If user handles the avcodec_get_hw_config() and set hw_frames_ctx, it will work. It doesn't work if
hw_frames_ctx is null, and hw_config is a capability and hint, not a hard requirement for all encoders.
Although we can enforce a valid hw_frames_ctx during videotoolbox encoder init, then it can break
use's code for no good reasons.

> 
> >>
> >>> ---
> >>>  libavcodec/videotoolboxenc.c | 9 +++++++++
> >>>  1 file changed, 9 insertions(+)
> >>>
> >>> diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c
> >>> index 1d1595329a..b685bf4cc4 100644
> >>> --- a/libavcodec/videotoolboxenc.c
> >>> +++ b/libavcodec/videotoolboxenc.c
> >>> @@ -37,6 +37,7 @@
> >>>  #include "encode.h"
> >>>  #include "h264.h"
> >>>  #include "h264_sei.h"
> >>> +#include "hwconfig.h"
> >>>  #include <dlfcn.h>
> >>>
> >>>  #if !HAVE_KCMVIDEOCODECTYPE_HEVC
> >>> @@ -2830,6 +2831,11 @@ static const enum AVPixelFormat prores_pix_fmts[] = {
> >>>          "Sets the maximum number of reference frames. This only has an effect when the value is less than the maximum
> allowed
> >> by the profile/level.", \
> >>>          OFFSET(max_ref_frames), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, INT_MAX, VE },
> >>>
> >>> +static const AVCodecHWConfigInternal *const vt_encode_hw_configs[] = {
> >>> +        HW_CONFIG_ENCODER_FRAMES(VIDEOTOOLBOX, VIDEOTOOLBOX),
> >>> +        NULL,
> >>> +};
> >>> +
> >>>  #define OFFSET(x) offsetof(VTEncContext, x)
> >>>  static const AVOption h264_options[] = {
> >>>      { "profile", "Profile", OFFSET(profile), AV_OPT_TYPE_INT, { .i64 = AV_PROFILE_UNKNOWN }, AV_PROFILE_UNKNOWN,
> INT_MAX,
> >> VE, "profile" },
> >>> @@ -2886,6 +2892,7 @@ const FFCodec ff_h264_videotoolbox_encoder = {
> >>>      .close            = vtenc_close,
> >>>      .p.priv_class     = &h264_videotoolbox_class,
> >>>      .caps_internal    = FF_CODEC_CAP_INIT_CLEANUP,
> >>> +    .hw_configs       = vt_encode_hw_configs,
> >>>  };
> >>>
> >>>  static const AVOption hevc_options[] = {
> >>> @@ -2923,6 +2930,7 @@ const FFCodec ff_hevc_videotoolbox_encoder = {
> >>>      .p.priv_class     = &hevc_videotoolbox_class,
> >>>      .caps_internal    = FF_CODEC_CAP_INIT_CLEANUP,
> >>>      .p.wrapper_name   = "videotoolbox",
> >>> +    .hw_configs       = vt_encode_hw_configs,
> >>>  };
> >>>
> >>>  static const AVOption prores_options[] = {
> >>> @@ -2961,4 +2969,5 @@ const FFCodec ff_prores_videotoolbox_encoder = {
> >>>      .p.priv_class     = &prores_videotoolbox_class,
> >>>      .caps_internal    = FF_CODEC_CAP_INIT_CLEANUP,
> >>>      .p.wrapper_name   = "videotoolbox",
> >>> +    .hw_configs       = vt_encode_hw_configs,
> >>>  };
> >>
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".