[FFmpeg-devel] [PATCH] avcodec: Add MediaFoundation encoder wrapper

[Martin Storsjö]

On Sun, 10 May 2020, James Almer wrote:

> On 5/9/2020 6:41 PM, Martin Storsjö wrote:
>> +#define MF_ENCODER(MEDIATYPE, NAME, ID, OPTS, EXTRA) \
>> +    static const AVClass ff_ ## NAME ## _mf_encoder_class = {                  \
>> +        .class_name = #NAME "_mf",                                             \
>> +        .item_name  = av_default_item_name,                                    \
>> +        .option     = OPTS,                                                    \
>> +        .version    = LIBAVUTIL_VERSION_INT,                                   \
>> +    };                                                                         \
>> +    AVCodec ff_ ## NAME ## _mf_encoder = {                                     \
>> +        .priv_class     = &ff_ ## NAME ## _mf_encoder_class,                   \
>> +        .name           = #NAME "_mf",                                         \
>> +        .long_name      = NULL_IF_CONFIG_SMALL(#ID " via MediaFoundation"),    \
>> +        .type           = AVMEDIA_TYPE_ ## MEDIATYPE,                          \
>> +        .id             = AV_CODEC_ID_ ## ID,                                  \
>> +        .priv_data_size = sizeof(MFContext),                                   \
>> +        .init           = mf_init,                                             \
>> +        .close          = mf_close,                                            \
>> +        .send_frame     = mf_send_frame,                                       \
>> +        .receive_packet = mf_receive_packet,                                   \
>> +        EXTRA                                                                  \
>> +        .capabilities   = AV_CODEC_CAP_DELAY,                                  \
>
> Should probably also be AV_CODEC_CAP_HYBRID.

Sure, added locally.

>> +        .caps_internal  = FF_CODEC_CAP_INIT_THREADSAFE |                       \
>> +                          FF_CODEC_CAP_INIT_CLEANUP,                           \
>> +    };
>> +
>> +#define AFMTS \
>> +        .sample_fmts    = (const enum AVSampleFormat[]){ AV_SAMPLE_FMT_S16,    \
>> +                                                         AV_SAMPLE_FMT_NONE },
>> +
>> +MF_ENCODER(AUDIO, aac,         AAC, NULL, AFMTS);
>> +MF_ENCODER(AUDIO, ac3,         AC3, NULL, AFMTS);
>> +MF_ENCODER(AUDIO, mp3,         MP3, NULL, AFMTS);
>> +
>> +#define VE AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_ENCODING_PARAM
>> +static const AVOption venc_opts[] = {
>> +    {"rate_control",  "Select rate control mode", OFFSET(opt_enc_rc), AV_OPT_TYPE_INT, {.i64 = -1}, -1, INT_MAX, VE, "rate_control"},
>> +    { "default",      "Default mode", 0, AV_OPT_TYPE_CONST, {.i64 = -1}, 0, 0, VE, "rate_control"},
>> +    { "cbr",          "CBR mode", 0, AV_OPT_TYPE_CONST, {.i64 = ff_eAVEncCommonRateControlMode_CBR}, 0, 0, VE, "rate_control"},
>> +    { "pc_vbr",       "Peak constrained VBR mode", 0, AV_OPT_TYPE_CONST, {.i64 = ff_eAVEncCommonRateControlMode_PeakConstrainedVBR}, 0, 0, VE, "rate_control"},
>> +    { "u_vbr",        "Unconstrained VBR mode", 0, AV_OPT_TYPE_CONST, {.i64 = ff_eAVEncCommonRateControlMode_UnconstrainedVBR}, 0, 0, VE, "rate_control"},
>> +    { "quality",      "Quality mode", 0, AV_OPT_TYPE_CONST, {.i64 = ff_eAVEncCommonRateControlMode_Quality}, 0, 0, VE, "rate_control" },
>> +    // The following rate_control modes require Windows 8.
>> +    { "ld_vbr",       "Low delay VBR mode", 0, AV_OPT_TYPE_CONST, {.i64 = ff_eAVEncCommonRateControlMode_LowDelayVBR}, 0, 0, VE, "rate_control"},
>> +    { "g_vbr",        "Global VBR mode", 0, AV_OPT_TYPE_CONST, {.i64 = ff_eAVEncCommonRateControlMode_GlobalVBR}, 0, 0, VE, "rate_control" },
>> +    { "gld_vbr",      "Global low delay VBR mode", 0, AV_OPT_TYPE_CONST, {.i64 = ff_eAVEncCommonRateControlMode_GlobalLowDelayVBR}, 0, 0, VE, "rate_control"},
>> +    {"quality",       "Quality", OFFSET(opt_enc_quality), AV_OPT_TYPE_INT, {.i64 = -1}, -1, 100, VE},
>> +    {"hw_encoding",   "Force hardware encoding", OFFSET(opt_enc_hw), AV_OPT_TYPE_BOOL, {.i64 = 0}, 0, 1, VE, "hw_encoding"},
>
> Can't you attempt to init using hw by default and if that fails
> gracefully fallback to the sw implementation?
> This option could instead just attempt to force disable or enable hw
> (deafault -1/auto), and if hw can't be used when forced it would just
> abort instead of falling back to sw.

It's not very straightforward to fall back, there's usually a rather long 
sequence of option setting until it's clear whether the chosen encoder 
works in the desired mode or not.

One common situation is e.g. that the system default SW encoder supports 
both yuv420p and nv12, while the HW encoder only supports nv12. Now if I'd 
prefer to use the hardware encoder, but it fails to use it (since the 
encoder turned out to only support nv12), falling back to SW would be a 
bit unexpected. (Although I guess one could retain that behaviour if the 
flag has been set to an explicit value.)

In any case, I'd rather leave this aspect as a patch welcome feature for 
others if they want to implement it - I don't see it as essential.

// Martin