[FFmpeg-devel] [PATCH] avfilter: add v360_vulkan filter

Thu Jul 4 17:47:55 EEST 2024

On 2024/06/25 22:27, Paul B Mahol wrote:
> On Tue, Jun 25, 2024 at 3:17 PM Dennis Mungai <dmngaie at gmail.com> wrote:
> 
>> On Tue, 25 Jun 2024 at 16:07, Paul B Mahol <onemda at gmail.com> wrote:
>>
>>> On Tue, Jun 25, 2024 at 2:47 PM Dennis Mungai <dmngaie at gmail.com> wrote:
>>>
>>>> On Sun, 28 Jun 2020 at 16:59, James Almer <jamrial at gmail.com> wrote:
>>>>
>>>>> On 6/27/2020 7:54 AM, Paul B Mahol wrote:
>>>>>> On 6/26/20, James Almer <jamrial at gmail.com> wrote:
>>>>>>> On 6/26/2020 4:16 PM, Lynne wrote:
>>>>>>>> Jun 26, 2020, 19:05 by jamrial at gmail.com:
>>>>>>>>
>>>>>>>>> On 6/26/2020 1:58 PM, Paul B Mahol wrote:
>>>>>>>>>
>>>>>>>>>> On 6/26/20, James Almer <jamrial at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> On 6/26/2020 1:13 PM, Paul B Mahol wrote:
>>>>>>>>>>>
>>>>>>>>>>>> On 6/26/20, Paul B Mahol <onemda at gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> On 6/24/20, Paul B Mahol <onemda at gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Signed-off-by: Paul B Mahol <onemda at gmail.com>
>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>> Please test it and reports benchmark results on non toyish GPUs.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Please, I really need it!
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Why nobody wants to help?
>>>>>>>>>>>>
>>>>>>>>>>>> This makes me really really sad.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I can try. Can you give an example command line with a publicly
>>>>>>>>>>> available sample?
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> You can use any sample, samples are irrelevant. just report resolution
>>>>>>>>>> used.
>>>>>>>>>> Also compare with CPU solution with just v360 filter.
>>>>>>>>>>
>>>>>>>>>> For yuv420p inputs:
>>>>>>>>>>
>>>>>>>>>> ffmpeg -init_hw_device vulkan=vulkan -i INPUT -filter_hw_device vulkan
>>>>>>>>>> -vf hwupload,v360_vulkan=fisheye:e:ih_fov=180:iv_fov=180:w=2048:h=1024,hwdownload,format=yuv420p -f null -
>>>>>>>>>>
>>>>>>>>>> ffmpeg -i INPUT -vf v360=fisheye:e:ih_fov=180:iv_fov=180:w=2048:h=1024 -f null -
>>>>>>>>>>
>>>>>>>>>> Just make sure that you get same output resolution in both cases. If
>>>>>>>>>> not, adjust w/h accordingly.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I'm getting a segfault on vkGetInstanceProcAddr, and i don't know if
>>>>>>>>> it's your filter or some issue in the driver. gdb is not being helpful.
>>>>>>>>> _______________________________________________
>>>>>>>>> ffmpeg-devel mailing list
>>>>>>>>> ffmpeg-devel at ffmpeg.org
>>>>>>>>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>>>>>>>>
>>>>>>>>> To unsubscribe, visit link above, or email
>>>>>>>>> ffmpeg-devel-request at ffmpeg.org>  with subject "unsubscribe".
>>>>>>>>>
>>>>>>>>
>>>>>>>> Can you test again but replace EXT_EXTERNAL_HOST_MEMORY in
>>>>>>>> optional_device_exts
>>>>>>>> with EXT_NO_FLAG?
>>>>>>>
>>>>>>> Applying
>>>>>>>
>>>>>>>> diff --git a/libavutil/hwcontext_vulkan.c b/libavutil/hwcontext_vulkan.c
>>>>>>>> index 5e51d0390f..49fb7827ba 100644
>>>>>>>> --- a/libavutil/hwcontext_vulkan.c
>>>>>>>> +++ b/libavutil/hwcontext_vulkan.c
>>>>>>>> @@ -228,7 +228,7 @@ static const VulkanOptExtension optional_device_exts[]
>>>>>>>> = {
>>>>>>>>       { VK_EXT_EXTERNAL_MEMORY_DMA_BUF_EXTENSION_NAME,
>>>>>>>> EXT_EXTERNAL_DMABUF_MEMORY, },
>>>>>>>>       { VK_EXT_IMAGE_DRM_FORMAT_MODIFIER_EXTENSION_NAME,
>>>>>>>> EXT_DRM_MODIFIER_FLAGS,     },
>>>>>>>>       { VK_KHR_EXTERNAL_SEMAPHORE_FD_EXTENSION_NAME,
>>>>>>>> EXT_EXTERNAL_FD_SEM,        },
>>>>>>>> -    { VK_EXT_EXTERNAL_MEMORY_HOST_EXTENSION_NAME,
>>>>>>>> EXT_EXTERNAL_HOST_MEMORY,   },
>>>>>>>> +    { VK_EXT_EXTERNAL_MEMORY_HOST_EXTENSION_NAME,
>>>>>>>> EXT_NO_FLAG,   },
>>>>>>>>   };
>>>>>>>>
>>>>>>>>   /* Converts return values to strings */
>>>>>>>
>>>>>>> I still get a segfault:
>>>>>>>
>>>>>>>> Thread 1 received signal SIGSEGV, Segmentation fault.
>>>>>>>> 0x00007ff8ed9274c0 in vkGetInstanceProcAddr ()
>>>>>>>>     from
>>>>>>>>
>>>>>
>>>>
>>>
>> C:\Windows\System32\DriverStore\FileRepository\nv_dispi.inf_amd64_ccad5caddc3a3d35\nvoglv64.dll
>>>>>>>
>>>>>>> Debug output shows
>>>>>>>
>>>>>>>> [AVHWDeviceContext @ 000001532a762800] GPU listing:
>>>>>>>> [AVHWDeviceContext @ 000001532a762800]     0: GeForce GTX 1660 (discrete)
>>>>>>>> (0x2184)
>>>>>>>> [AVHWDeviceContext @ 000001532a762800] Using queue family 0 (total queues:
>>>>>>>> 16) for graphics
>>>>>>>> [AVHWDeviceContext @ 000001532a762800]     QF 0 flags: (graphics)
>>>>>>>> (compute) (transfers) (sparse)
>>>>>>>> [AVHWDeviceContext @ 000001532a762800] Using queue family 2 (total queues:
>>>>>>>> 8) for compute
>>>>>>>> [AVHWDeviceContext @ 000001532a762800]     QF 2 flags: (compute)
>>>>>>>> (transfers) (sparse)
>>>>>>>> [AVHWDeviceContext @ 000001532a762800] Using queue family 1 (total queues:
>>>>>>>> 2) for transfers
>>>>>>>> [AVHWDeviceContext @ 000001532a762800]     QF 1 flags: (transfers)
>>>>>>>> (sparse)
>>>>>>>> [AVHWDeviceContext @ 000001532a762800] Using device extension
>>>>>>>> "VK_EXT_external_memory_host"
>>>>>>>> [AVHWDeviceContext @ 000001532a762800] Using device extension
>>>>>>>> VK_EXT_external_memory_host
>>>>>>>> [AVHWDeviceContext @ 000001532a762800] Using device: GeForce GTX 1660
>>>>>>>> [AVHWDeviceContext @ 000001532a762800] Alignments:
>>>>>>>> [AVHWDeviceContext @ 000001532a762800]
>>>>>>>> optimalBufferCopyRowPitchAlignment: 1
>>>>>>>> [AVHWDeviceContext @ 000001532a762800]     minMemoryMapAlignment:      64
>>>>>>>
>>>>>>> And crashes right after printing:
>>>>>>>
>>>>>>>> [Parsed_v360_vulkan_1 @ 000001532abd5c80] Shader v360_compute linked!
>>>>>>>> Size: 5508 bytes
>>>>>>>
>>>>>>> I'm using the latest driver, 451.48, on Windows 10 2004.
>>>>>>
>>>>>> So other vulkan filters, like vulkan_chromaber works for you?
>>>>>
>>>>> All fail the same way in vkGetInstanceProcAddr, so the issue is in
>>>>> hwcontext.
>>>>>
>>>>> Lynne should be able to test/bench this filter.
>>>>>
>>>>
>>>>   Hello Paul, et al,
>>>>
>>>> Any updates on this filter?
>>>>
>>>
>>> I'm no longer working on FFmpeg.
>>>
>>> The v360_opencl filter got in more useful state but due complete lack of
>>> interest in it from users, I forgot about its existence.
>>> The v360_vulkan filter is probably in worse shape than v360_opencl filter
>>> and 0/negative interest, and I forgot about its existence too.
>>>
>>
>> Is there an FFmpeg tree with v360_opencl (or a similar filter) out there?
>>
> 
> 
> I can give you patches/files for both v360 vulkan and opencl filter if you
> want to hack on it.
> But I will not adapt them for free to current library versions, nor provide
> any kind of free support for it.
> 

I tested v360_opencl filter with slight modification to your patch.
It works, but it generates different output from v360 filter.

For example, I tested a sample GoPro Max 360 video on:
https://drive.google.com/drive/folders/1DlP6FVw-O6AIm6O0nSsC-mIoCrEBtmdG

ffmpeg -i GS010010.360 -init_hw_device opencl:0.0 -vf "hwupload,v360_opencl=yaw=90,hwdownload,format=yuvj420p" output-v360cl.mkv
ffmpeg -i GS010010.360 -vf v360=yaw=90 output-v360.mkv

I'm not sure which is correct, but I feel the output of v360_opencl filter is better.

> 
>> A GPU-accelerated v360 filter is definitely useful to test and implement.
>> Definitely more useful than the DNN stack in FFmpeg (by leaps and bounds)
>> as far as applicability is concerned.
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel at ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".