[FFmpeg-devel] [PATCH 1/4] kmsgrab: Fix 32-bit RGB DRM format definitions

[Carl Eugen Hoyos]

2017-09-15 23:44 GMT+02:00 Mark Thompson <sw at jkqxz.net>:
> On 15/09/17 22:40, Carl Eugen Hoyos wrote:
>> 2017-09-15 22:51 GMT+02:00 Mark Thompson <sw at jkqxz.net>:
>>> The 32-bit DRM formats are defined in terms of little-endian words, so
>>> 32-bit RGB formats like XRGB actually have the elements in the opposite
>>> order in memory to the order they are in the name.
>>>
>>> This does not affect YUYV and similar YUV 4:2:2 formats, which are in
>>> the expected order.
>>> ---
>>>  libavdevice/kmsgrab.c | 16 ++++++++--------
>>>  1 file changed, 8 insertions(+), 8 deletions(-)
>>>
>>> diff --git a/libavdevice/kmsgrab.c b/libavdevice/kmsgrab.c
>>> index 67a83ef84a..bcb6865f60 100644
>>> --- a/libavdevice/kmsgrab.c
>>> +++ b/libavdevice/kmsgrab.c
>>> @@ -210,14 +210,14 @@ static const struct {
>>>  #endif
>>>      { AV_PIX_FMT_RGB24,    DRM_FORMAT_RGB888   },
>>>      { AV_PIX_FMT_BGR24,    DRM_FORMAT_BGR888   },
>>> -    { AV_PIX_FMT_0RGB,     DRM_FORMAT_XRGB8888 },
>>> -    { AV_PIX_FMT_0BGR,     DRM_FORMAT_XBGR8888 },
>>> -    { AV_PIX_FMT_RGB0,     DRM_FORMAT_RGBX8888 },
>>> -    { AV_PIX_FMT_BGR0,     DRM_FORMAT_BGRX8888 },
>>> -    { AV_PIX_FMT_ARGB,     DRM_FORMAT_ARGB8888 },
>>> -    { AV_PIX_FMT_ABGR,     DRM_FORMAT_ABGR8888 },
>>> -    { AV_PIX_FMT_RGBA,     DRM_FORMAT_RGBA8888 },
>>> -    { AV_PIX_FMT_BGRA,     DRM_FORMAT_BGRA8888 },
>>> +    { AV_PIX_FMT_0RGB,     DRM_FORMAT_BGRX8888 },
>>> +    { AV_PIX_FMT_0BGR,     DRM_FORMAT_RGBX8888 },
>>> +    { AV_PIX_FMT_RGB0,     DRM_FORMAT_XBGR8888 },
>>> +    { AV_PIX_FMT_BGR0,     DRM_FORMAT_XRGB8888 },
>>> +    { AV_PIX_FMT_ARGB,     DRM_FORMAT_BGRA8888 },
>>> +    { AV_PIX_FMT_ABGR,     DRM_FORMAT_RGBA8888 },
>>> +    { AV_PIX_FMT_RGBA,     DRM_FORMAT_ABGR8888 },
>>> +    { AV_PIX_FMT_BGRA,     DRM_FORMAT_ARGB8888 },
>>
>> Is it possible to compile kmsgrab on big-endian hardware?
>
> Yes.

And you assume that on big-endian hardware, above formats
would still be little-endian?

Or that they would not be used at all and the big-endian bit would
be set?
In that case, it may be simpler to mask the most significant
bit away in the code and use the endianness-aware formats
for FFmpeg in the table above (it would reduce the table size
and make it more readable). Especially as we don't know if
the bit would also be set for the packed yuv formats.

Carl Eugen