[FFmpeg-devel] [PATCH v4 1/1] avutils/hwcontext: When deriving a hwdevice, search for existing device in both directions

[Mark Thompson]

On 10/01/2022 01:40, Soft Works wrote:
>> -----Original Message-----
>> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of Mark
>> Thompson
>> Sent: Monday, January 10, 2022 1:57 AM
>> To: ffmpeg-devel at ffmpeg.org
>> Subject: Re: [FFmpeg-devel] [PATCH v4 1/1] avutils/hwcontext: When deriving a
>> hwdevice, search for existing device in both directions
>>
>> [trimmed somewhat]
>>
>> Device derivation means making a compatible context of a different type on
>> the same physical device.
>>
>> Now that's probably intended because you are going to want to share some
>> particular resources, but exactly what can be shared and what is possible is
>> dependent on the setup.
>>
>> Similarly, any rules for concurrency are dependent on the setup - maybe you
>> can't do two specific things simultaneously in the same device context and
>> need two separate ones to solve it, but they still both want to share in some
>> way with the different device context they were derived from.
>>
>>>> * When global options have to be set on a device, so a component which
>> does
>>>> that needs its own instance to avoid interfering with anyone else.
>>>
>>> This is NOT derivation. This case is not affected.
>>
>> Suppose I have some DRM frames which come from somewhere (some hardware
>> decoder, say - doesn't matter).
>>
>> I want to do Vulkan things with some of the frames, so I call
>> av_hwdevice_ctx_create_derived_opts() to get a compatible Vulkan context.
>> Then I can map and do Vulkan things, yay!
>>
>> Later on, I want to do some independent Vulkan thing with my DRM frames.  I
>> do the same operation again with different options (because my new thing
>> wants some new extensions, say).  This returns a new Vulkan context and I can
>> work with it completely independently, yay!
> 
> You are describing the creation of a Vulkan context with other parameters
> with which you can work independently.
> 
> That's not my understanding of deriving a context. I don't the implementation
> in case of Vulkan, but in case of the others, deriving is about sharing
> resources. And when you share resources, you can't "work with it independently",
> so what you're talking about is not a scenario of deriving a context.

But that's what deriving a context is as currently implemented.

In my example, the Vulkan parts don't care about where the context came from.  Derivation is used to ensure that frames can be shared, but all compute happens independently.

> To wrap things up a bit:
> 
> - you want an approach which requires even more complicated filter
>    command lines.

Ha, that characterisation isn't exactly neutral - the derivation in filter graphs is removed and replaced with explicit specification of devices.

Indeed, once device creation is explicit, command lines are clearer because you don't need to reason about creation of devices inside filter graphs and whether they are actually the same device.

>    I have understood that. It is a possible addition for filter command lines
>    and I would even welcome somebody who would implement precise hw context
>    selection for hwdownload  and also for hwmapn for (rare) cases where this
>    might be needed. (that somebody won't be me, though)
> 
> - but this is not what this patchset is about. it is about having things
>    working nicely and automatically in a way as one would expect it instead
>    of failing. this patchset only touches and changes behavior in those cases
>    that are currently failing anyway

No it doesn't; it changes the behaviour of a library API when you are talking about use-cases in the ffmpeg utility.

If you need some library assistance, perhaps by making some new concept of a bundle of related devices which have the properties you want, then maybe that's a good idea.

Perhaps you even want to add to the av_hwdevice_ctx_create_derived() API so it can take a YKNOW_ACTUALLY_DONT flag to indicate you don't want to create a new derived device, I don't know.

- Mark