[FFmpeg-devel] [PATCH 3/8] lavu: add a Vulkan hwcontext

[Rostislav Pehlivanov]

On 22 April 2018 at 17:59, Rostislav Pehlivanov <atomnuker at gmail.com> wrote:

>
>
> On 22 April 2018 at 17:51, Mark Thompson <sw at jkqxz.net> wrote:
>
>> On 22/04/18 17:36, Mark Thompson wrote:
>> > On 22/04/18 17:28, Rostislav Pehlivanov wrote:
>> >> On 22 April 2018 at 17:21, Rostislav Pehlivanov <atomnuker at gmail.com>
>> wrote:
>> >>> On 22 April 2018 at 12:46, Mark Thompson <sw at jkqxz.net> wrote:
>> >>>> On 21/04/18 23:29, Carl Eugen Hoyos wrote:
>> >>>>> 2018-04-21 23:33 GMT+02:00, Rostislav Pehlivanov <
>> atomnuker at gmail.com>:
>> >>>>>> On 21 April 2018 at 21:24, Carl Eugen Hoyos <ceffmpeg at gmail.com>
>> >>>> wrote:
>> >>>>>>
>> >>>>>>> 2018-04-20 6:30 GMT+02:00, Rostislav Pehlivanov <
>> atomnuker at gmail.com
>> >>>>> :
>> >>>>>>>
>> >>>>>>>> +    [AV_PIX_FMT_P010]      =
>> >>>>>>>> VK_FORMAT_G10X6_B10X6R10X6_2PLANE_420_UNORM_3PACK16,
>> >>>>>>>
>> >>>>>>>> +    [AV_PIX_FMT_YUV420P10] =
>> >>>>>>>> VK_FORMAT_G10X6_B10X6_R10X6_3PLANE_420_UNORM_3PACK16,
>> >>>>>>>
>> >>>>>>> I don't think both can be correct (unless "PACK16" has no
>> meaning).
>> >>>>>
>> >>>>>> They're both correct and work.
>> >>>>>
>> >>>>> That's really strange...
>> >>>>> (Could this be a bug in the driver?)
>> >>>>
>> >>>> Sounds like it must be a bug somewhere.
>> >>>>
>> >>>> The Vulkan specification says:
>> >>>>
>> >>>> """
>> >>>> VK_FORMAT_G10X6_B10X6R10X6_2PLANE_420_UNORM_3PACK16 specifies a
>> unsigned
>> >>>> normalized multi-planar format that has a 10-bit G component in the
>> top 10
>> >>>> bits of each 16-bit word of plane 0, and a two-component, 32-bit BR
>> plane 1
>> >>>> consisting of a 10-bit B component in the top 10 bits of the word in
>> bytes
>> >>>> 0..1, and a 10-bit R component in the top 10 bits of the word in
>> bytes
>> >>>> 2..3, the bottom 6 bits of each word set to 0.
>> >>>>
>> >>>> VK_FORMAT_G10X6_B10X6_R10X6_3PLANE_420_UNORM_3PACK16 specifies a
>> >>>> unsigned normalized multi-planar format that has a 10-bit G
>> component in
>> >>>> the top 10 bits of each 16-bit word of plane 0, a 10-bit B component
>> in the
>> >>>> top 10 bits of each 16-bit word of plane 1, and a 10-bit R component
>> in the
>> >>>> top 10 bits of each 16-bit word of plane 2, with the bottom 6 bits
>> of each
>> >>>> word set to 0.
>> >>>> """
>> >>>>
>> >>>> Which I think makes it pretty clear that
>> VK_FORMAT_G10X6_B10X6R10X6_2PLANE_420_UNORM_3PACK16
>> >>>> is indeed P010 but VK_FORMAT_G10X6_B10X6_R10X6_3P
>> LANE_420_UNORM_3PACK16
>> >>>> isn't YUV420P10 because they pack the 10 bits at different ends of
>> the
>> >>>> 16-bit value.  If a driver is getting that wrong then it should be
>> reported
>> >>>> to the vendor.
>> >>>>
>> >>>> I don't see any formats at all in the Vulkan specification which put
>> the
>> >>>> value at the low end of a containing word, but I might not be
>> looking in
>> >>>> the right place?
>> >>>>
>> >>>> - Mark
>> >>>>
>> >>>>
>> >>>> (Vaguely related, because it made me look it up, it appears that the
>> >>>> device will always match host-endianness:
>> >>>>
>> >>>> After talking about the numeric types,
>> >>>> """
>> >>>> The representation and endianness of these types on the host must
>> match
>> >>>> the representation and endianness of the same types on every physical
>> >>>> device supported."
>> >>>> """
>> >>>>
>> >>>> I don't know what that actually means for little-endian graphics
>> cards
>> >>>> (e.g. AMD/Nvidia) in big-endian machines (e.g. POWER) - maybe Vulkan
>> just
>> >>>> doesn't support that, or maybe the driver can fix it up somehow -
>> but we
>> >>>> don't need to think about it at all.)
>> >>>
>> >>> Something's weird:
>> >>>
>> >>>
>> >> Sorry, pushed the wrong button.
>> >>
>> >> For this filter chain:
>> >> format=<SRC_FORMAT>,hwupload,scale_vulkan=w=1024:h=-1:format
>> =rgba,hwdownload,format=rgba
>> >>
>> >> This is what happens for each SRC_FORMAT:
>> >> NVIDIA 960M with binary drivers:
>> >> p010 - works fine
>> >> yuv420p10le - mostly green screen with some minor variations, enough to
>> >> make out the original video
>> >> yuv420p10be - works fine
>> >>
>> >> Intel 530:
>> >> p010 - works fine
>> >> yuv420p10le - works fine
>> >> yuv420p10be - works fine
>> >>
>> >> I'm not entirely sure what to make of that. How does the intel deal
>> with
>> >> formats with different endianess when there's no way to indicate
>> endianess
>> >> at all? Why does nvidia deal with big endian when you said its little
>> >> endian?
>> >
>> > hwupload checks the supported formats with get_constraints and only
>> exposes the supported ones to lavfi query_formats.  Probably some
>> auto-conversion is happening for the big-endian formats?  Maybe on Intel
>> the three-plane format also isn't supported, and so auto-conversion happens
>> there too?
>> >
>> > I think the green screen is what we would expect from the above
>> analysis, since all you would be getting is the high 4 bits of each
>> component in the low 4 bits of the output.
>>
>> Assuming that by Intel you mean Mesa rather than Windows blob, <
>> https://cgit.freedesktop.org/mesa/mesa/tree/src/intel/vulka
>> n/anv_formats.c#n329> says that none of these formats are supported
>> (P010 or YUV420P10).  On that driver it would be converting to something
>> else in software for all of them.
>>
>> - Mark
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel at ffmpeg.org
>> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
>
> Ah, seems like it does.
> I've locally removed YUV420P10 from the list.
>

I looked over the supported formats, it seems all 4XXX-10 and 12 bit ones
were defined as bottom-zeroe'd. So removed AV_PIX_FMT_YUV420P10,
AV_PIX_FMT_YUV420P12, AV_PIX_FMT_YUV422P10, AV_PIX_FMT_YUV422P12,
AV_PIX_FMT_YUV444P10, AV_PIX_FMT_YUV444P12.

All the other 3-plane ones fit in exactly 8 or 16 bits.