[FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to use initialized device

[Mark Thompson]

On 02/09/2020 15:36, Rogozhkin, Dmitry V wrote:
> On Wed, 2020-09-02 at 14:21 +0000, Rogozhkin, Dmitry V wrote:
>> On Wed, 2020-09-02 at 08:41 +0000, Soft Works wrote:
>>> ...
> 
> Small suggestion: let's move discussion around -qsv_device and
> -hwaccel_device options entirely to the "ffmpeg_qsv: use
> -hwaccel_device to specify a device for VAAPI backend" thread and a
> return focus on the original patch which is not about -qsv_device, but
> about this command line:
> 
> ffmpeg -init_hw_device vaapi=va:/dev/dri/renderD129 -init_hw_device
> qsv=hw at va -c:v h264_qsv -i input.h264 -c:v hevc_qsv -y output.h264

I wouldn't recommend doing this; using the appropriate acceleration API is preferred to messing with the vendor-specific full-offload decoders:

ffmpeg -init_hw_device vaapi:/dev/dri/renderD129 -hwaccel vaapi -i input.h264 -vf hwmap=derive_device=qsv -c:v hevc_qsv -y output.h264

(Or ... -init_hw_device dxva2:1 -hwaccel dxva2 ...)

> For what I see it uses valid device specifications and still in
> original ffmpeg implementation it results in:
> - decoder is being run on /dev/dri/renderD128
> - encoder is being run on /dev/dri/renderD129

The libmfx decoders need you to set -qsv_device, which is a hack option for ffmpeg to allow selecting a device on them because they doesn't support the normal device selection.

The libmfx encoders do support the normal setup in libavcodec, so ffmpeg matched hevc_qsv to the libmfx device you created.

> In general, per my taste, I would try to use the following device
> specification working with QSV on Linux across all command lines:
> 
> -init_hw_device vaapi=va:/dev/dri/renderD129 -init_hw_device qsv=hw at va
> -filter_hw_device hw
> 
> My primary goal is to make it workable for all pipelines. Hence current
> patch.

Implementing AV_HW_CONFIG_METHOD_HW_DEVICE_CTX would make sense to allow the legacy decoders to use normal device selection, but your patch seems to just set the flag indicating support without actually implementing the feature?

(If you do implement it then you can delete all of the ad-hoc treatment in ffmpeg, like has been done for the other hardware codecs.)

>>> ...
>>
>> I still don't see the full picture. What I am looking for at the
>> first
>> place is how maintainers and architects envision hwaccel to work in
>> general. Basically, there are few ways to specify the device (-
>> hwaccel,
>> -init_hw_device, -filter_hw_device), questions are:
>> 1. What's scope of applicability of each option w/ explanation why
>> each
>> of the option is actually needed (as one of the examples: why -
>> filter_hw_device is needed and why -init_hw_device can't be used
>> instead?
-init_hw_device is a global option which defines a device and adds it to the list in ffmpeg; you can have as many of these as you like.

When setting up a codec which might be able to use a device it tries to match against that list, unless you have an explicit -hwaccel_device option for that input.  If you do, then it looks for the device with that name.

To maintain compatibility with old command-lines where -hwaccel_device took a string interpreted by special code for each API in ffmpeg (now removed), if it doesn't find a device with the given name then it will try to use it to create a new one.

(So "-hwaccel foo -hwaccel_device bar" when "bar" hasn't been previously defined ends up with the effect of "-init_hw_device foo=quux:bar -hwaccel foo -hwaccel_device quux".)

This doesn't apply to the libmfx decoders, because they have their own ad-hoc code.  Getting rid of that so they work in the same way as everything else would be a good thing.

-filter_hw_device selects a device from the list to use while filtering (e.g. by hwupload).

If there is exactly one device in the list when making a filter graph then it will use that, but if there are multiple then it has no way to guess which one you meant so the option is needed.

>> 2. Since there are few methods how component can get a device: what's
>> the priority order? (for example, if device can be deducted from
>> incoming frames, is there a way to override it w/ some command line
>> options?)

Devices from incoming frames have to win, because they are stored on that device - it doesn't make sense to select a different device at that point (you would need a separate download/upload or mapping step to do that).

- Mark