[FFmpeg-devel] [PATCH 31/92] Vulkan patchset part 1 - common code changes

[Lynne]

Apr 22, 2023, 14:42 by anton at khirnov.net:

> Quoting Lynne (2023-04-19 16:39:28)
>
>> Apr 17, 2023, 11:25 by anton at khirnov.net:
>>
>> > Quoting Lynne (2023-03-14 07:33:43)
>> >
>> >> From 6e2dfc44e50798264eb16bc2dcabfdbf2fbac2d7 Mon Sep 17 00:00:00 2001
>> >> From: Lynne <dev at lynne.ee>
>> >> Date: Thu, 15 Dec 2022 01:06:52 +0100
>> >> Subject: [PATCH 24/92] hwconfig: add a new HWACCEL_CAP_THREAD_SAFE for
>> >>  threadsafe hwaccels
>> >>
>> >> Vulkan is fully threadsafe and stateless, so we can benefit from this.
>> >> ---
>> >>  libavcodec/hwconfig.h      | 1 +
>> >>  libavcodec/pthread_frame.c | 2 +-
>> >>  2 files changed, 2 insertions(+), 1 deletion(-)
>> >>
>> >> diff --git a/libavcodec/hwconfig.h b/libavcodec/hwconfig.h
>> >> index 721424912c..e6b78f0160 100644
>> >> --- a/libavcodec/hwconfig.h
>> >> +++ b/libavcodec/hwconfig.h
>> >> @@ -24,6 +24,7 @@
>> >> 
>> >> 
>> >>  #define HWACCEL_CAP_ASYNC_SAFE      (1 << 0)
>> >> +#define HWACCEL_CAP_THREAD_SAFE     (1 << 1)
>> >> 
>> >> 
>> >>  typedef struct AVCodecHWConfigInternal {
>> >> diff --git a/libavcodec/pthread_frame.c b/libavcodec/pthread_frame.c
>> >> index 74864e19c5..c096287233 100644
>> >> --- a/libavcodec/pthread_frame.c
>> >> +++ b/libavcodec/pthread_frame.c
>> >> @@ -204,7 +204,7 @@ static attribute_align_arg void *frame_worker_thread(void *arg)
>> >> 
>> >>  /* if the previous thread uses hwaccel then we take the lock to ensure
>> >>  * the threads don't run concurrently */
>> >> -        if (avctx->hwaccel) {
>> >> +        if (avctx->hwaccel && !(avctx->hwaccel->caps_internal & HWACCEL_CAP_THREAD_SAFE)) {
>> >>
>> >
>> > A major problem here is that hwaccel_priv_data exists in only a single
>> > instance shared across all the worker threads. If you want frame-threaded
>> > hwaccels, you'll need to add some mechanism for synchronizing it between
>> > threads.
>> >
>> > I really wonder how does your code survive all the races involved.
>> >
>>
>> It works because hwaccel_priv_data is read-only. No need to synchronize
>> anything, the contexts Vulkan creates are defined as driver-synchronized,
>> and all the info a frame needs is provided when decoding it, rather than
>> kept as a state.
>> So I don't see a problem here.
>>
>
> From a quick glance, at least vk_av1_free_frame_priv() modifies this
> context, which seems highly unsafe.
>

Our AV1 decoder isn't threaded yet. I've removed the threadsafe flag from the
hwaccel, Dave is currently looking into whether it's possible to remove the
frame id.


> This really seems like a disaster waiting to happen, there should be
> some stronger guarantees that nobody will modify this context when they
> shouldn't.
>

It's an opt-in flag, that's more than enough.