[FFmpeg-devel] [PATCH 3/8] lavu: add a Vulkan hwcontext

[Mark Thompson]

On 22/04/18 17:36, Mark Thompson wrote:
> On 22/04/18 17:28, Rostislav Pehlivanov wrote:
>> On 22 April 2018 at 17:21, Rostislav Pehlivanov <atomnuker at gmail.com> wrote:
>>> On 22 April 2018 at 12:46, Mark Thompson <sw at jkqxz.net> wrote:
>>>> On 21/04/18 23:29, Carl Eugen Hoyos wrote:
>>>>> 2018-04-21 23:33 GMT+02:00, Rostislav Pehlivanov <atomnuker at gmail.com>:
>>>>>> On 21 April 2018 at 21:24, Carl Eugen Hoyos <ceffmpeg at gmail.com>
>>>> wrote:
>>>>>>
>>>>>>> 2018-04-20 6:30 GMT+02:00, Rostislav Pehlivanov <atomnuker at gmail.com
>>>>> :
>>>>>>>
>>>>>>>> +    [AV_PIX_FMT_P010]      =
>>>>>>>> VK_FORMAT_G10X6_B10X6R10X6_2PLANE_420_UNORM_3PACK16,
>>>>>>>
>>>>>>>> +    [AV_PIX_FMT_YUV420P10] =
>>>>>>>> VK_FORMAT_G10X6_B10X6_R10X6_3PLANE_420_UNORM_3PACK16,
>>>>>>>
>>>>>>> I don't think both can be correct (unless "PACK16" has no meaning).
>>>>>
>>>>>> They're both correct and work.
>>>>>
>>>>> That's really strange...
>>>>> (Could this be a bug in the driver?)
>>>>
>>>> Sounds like it must be a bug somewhere.
>>>>
>>>> The Vulkan specification says:
>>>>
>>>> """
>>>> VK_FORMAT_G10X6_B10X6R10X6_2PLANE_420_UNORM_3PACK16 specifies a unsigned
>>>> normalized multi-planar format that has a 10-bit G component in the top 10
>>>> bits of each 16-bit word of plane 0, and a two-component, 32-bit BR plane 1
>>>> consisting of a 10-bit B component in the top 10 bits of the word in bytes
>>>> 0..1, and a 10-bit R component in the top 10 bits of the word in bytes
>>>> 2..3, the bottom 6 bits of each word set to 0.
>>>>
>>>> VK_FORMAT_G10X6_B10X6_R10X6_3PLANE_420_UNORM_3PACK16 specifies a
>>>> unsigned normalized multi-planar format that has a 10-bit G component in
>>>> the top 10 bits of each 16-bit word of plane 0, a 10-bit B component in the
>>>> top 10 bits of each 16-bit word of plane 1, and a 10-bit R component in the
>>>> top 10 bits of each 16-bit word of plane 2, with the bottom 6 bits of each
>>>> word set to 0.
>>>> """
>>>>
>>>> Which I think makes it pretty clear that VK_FORMAT_G10X6_B10X6R10X6_2PLANE_420_UNORM_3PACK16
>>>> is indeed P010 but VK_FORMAT_G10X6_B10X6_R10X6_3PLANE_420_UNORM_3PACK16
>>>> isn't YUV420P10 because they pack the 10 bits at different ends of the
>>>> 16-bit value.  If a driver is getting that wrong then it should be reported
>>>> to the vendor.
>>>>
>>>> I don't see any formats at all in the Vulkan specification which put the
>>>> value at the low end of a containing word, but I might not be looking in
>>>> the right place?
>>>>
>>>> - Mark
>>>>
>>>>
>>>> (Vaguely related, because it made me look it up, it appears that the
>>>> device will always match host-endianness:
>>>>
>>>> After talking about the numeric types,
>>>> """
>>>> The representation and endianness of these types on the host must match
>>>> the representation and endianness of the same types on every physical
>>>> device supported."
>>>> """
>>>>
>>>> I don't know what that actually means for little-endian graphics cards
>>>> (e.g. AMD/Nvidia) in big-endian machines (e.g. POWER) - maybe Vulkan just
>>>> doesn't support that, or maybe the driver can fix it up somehow - but we
>>>> don't need to think about it at all.)
>>>
>>> Something's weird:
>>>
>>>
>> Sorry, pushed the wrong button.
>>
>> For this filter chain:
>> format=<SRC_FORMAT>,hwupload,scale_vulkan=w=1024:h=-1:format=rgba,hwdownload,format=rgba
>>
>> This is what happens for each SRC_FORMAT:
>> NVIDIA 960M with binary drivers:
>> p010 - works fine
>> yuv420p10le - mostly green screen with some minor variations, enough to
>> make out the original video
>> yuv420p10be - works fine
>>
>> Intel 530:
>> p010 - works fine
>> yuv420p10le - works fine
>> yuv420p10be - works fine
>>
>> I'm not entirely sure what to make of that. How does the intel deal with
>> formats with different endianess when there's no way to indicate endianess
>> at all? Why does nvidia deal with big endian when you said its little
>> endian?
> 
> hwupload checks the supported formats with get_constraints and only exposes the supported ones to lavfi query_formats.  Probably some auto-conversion is happening for the big-endian formats?  Maybe on Intel the three-plane format also isn't supported, and so auto-conversion happens there too?
> 
> I think the green screen is what we would expect from the above analysis, since all you would be getting is the high 4 bits of each component in the low 4 bits of the output.

Assuming that by Intel you mean Mesa rather than Windows blob, <https://cgit.freedesktop.org/mesa/mesa/tree/src/intel/vulkan/anv_formats.c#n329> says that none of these formats are supported (P010 or YUV420P10).  On that driver it would be converting to something else in software for all of them.

- Mark