[FFmpeg-devel] [PATCH v5 5/5] lavfi/format: add a hwmap auto conversion filter

[Wu, Tong1]

>On 25 Apr 2023, at 9:26, Tong Wu wrote:
>
>> When two formats lists cannot be merged, a scale filter is
>> auto-inserted. However, when it comes to hardware map, we have to
>> manually add a hwmap filter to do the conversion. This patch introduces
>> an auto hwmap filter to do the hwmap conversion automatically.
>>
>
>Thanks for trying to improve this!
>
>I've recently done quite a bit of experimentation with hardware
>filters and at least for the Cuda - Vulkan - Cuda case, hwmap
>was useless, and I was told I need to use hwupload instead, so I wonder
>what cases this would help with?

Thanks for the reply!

Actually we are using hwmap filter to do vaapi<->qsv, d3d11<->qsv conversions.

For the use case I have replied in another mail. I'll just copy here again.
For example,
ffmpeg -init_hw_device d3d11va=d3d11 -init_hw_device qsv=qsv at d3d11 -
hwaccel d3d11va -hwaccel_output_format d3d11 -i input.mp4 -vf
"hwmap=derive_device=qsv:extra_hw_frames=16,format=qsv" -c:v h264_qsv
output.mp4

Now we don't have to explicitly specify hwmap. The auto filter mechanism will
try to insert a hwmap filter automatically. The command line will be much
simpler.

ffmpeg -init_hw_device d3d11va=d3d11 -init_hw_device qsv=qsv at d3d11 -
hwaccel d3d11va -hwaccel_output_format d3d11 -i input.mp4 -c:v h264_qsv
output.mp4



>
>I just fear that, especially given the bad error messages hwmap gives,
>this will just implicitly insert it because it seemingly works but then
>just fail to actually do the job and give an absolutely indescriptive
>error to the user for a filter they did not even insert themselves.

Yes I agree with that. However, we already have an auto scale filter in the framework.
It will always be inserted when two filters cannot be linked.

In my opinion, I'm just kinda appending another filter. It should be no harmful to the original framework. The only way you can get to this auto hwmap filter is that you already fail to insert scale filter and the program will return error. That's the current situation. With this patch set, those failed cmdlines could still fail but some cmdlines like above could pass. It just give it another chance to run and simplify the cmdlines.

Regards,
Tong


>
>> Signed-off-by: Tong Wu <tong1.wu at intel.com>
>> ---
>>  libavfilter/avfiltergraph.c | 3 ++-
>>  libavfilter/formats.c       | 4 ++++
>>  2 files changed, 6 insertions(+), 1 deletion(-)
>>
>> diff --git a/libavfilter/avfiltergraph.c b/libavfilter/avfiltergraph.c
>> index 8af0467bc5..b2b627ad6a 100644
>> --- a/libavfilter/avfiltergraph.c
>> +++ b/libavfilter/avfiltergraph.c
>> @@ -402,7 +402,8 @@ static int insert_auto_filter(AVFilterContext
>**convert, AVFilterGraph *graph,
>>      AVFilterContext *ctx;
>>      AVFilterLink *inlink, *outlink;
>>      char inst_name[30];
>> -    const char *opts = FF_FIELD_AT(char *, neg-
>>conversion_filters[conv_step].conversion_opts_offset, *graph);
>> +    const char *opts = neg-
>>conversion_filters[conv_step].conversion_opts_offset == 0 ? NULL :
>> +                       FF_FIELD_AT(char *, neg-
>>conversion_filters[conv_step].conversion_opts_offset, *graph);
>>      const char *name = neg->conversion_filters[conv_step].conversion_filter;
>>
>>      if (!(filter = avfilter_get_by_name(name))) {
>> diff --git a/libavfilter/formats.c b/libavfilter/formats.c
>> index c8e20e5b20..fee10fa0ee 100644
>> --- a/libavfilter/formats.c
>> +++ b/libavfilter/formats.c
>> @@ -331,6 +331,10 @@ static const AVFilterFormatsFilter filters_video[] = {
>>          .conversion_filter = "scale",
>>          .conversion_opts_offset = offsetof(AVFilterGraph, scale_sws_opts),
>>      },
>> +    {
>> +        .conversion_filter = "hwmap",
>> +        .conversion_opts_offset = 0,
>> +    }
>>  };
>>
>>  static const AVFilterFormatsFilter filters_audio[] = {
>> --
>> 2.39.1.windows.1
>>
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel at ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>_______________________________________________
>ffmpeg-devel mailing list
>ffmpeg-devel at ffmpeg.org
>https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>To unsubscribe, visit link above, or email
>ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".