[FFmpeg-devel] [PATCH 0/4] Add AVDRMFrameDescriptor.format field

[Mark Thompson]

On 09/05/2019 20:38, Jonas Karlman wrote:
> Hello,
> 
> When a multi-layer AVDRMFrameDescriptor is used to describe a frame the overall
> frame format is missing and applications need to deduce the frame DRM_FORMAT_*
> based on sw_format or the layers format.
> 
> This patchset adds a AVDRMFrameDescriptor.format field to remove any ambiguity
> of what frame format a multi-layer descriptor may have.
> 
> Kodi has up until now only supported single layer AVDRMFrameDescriptor,
> when trying to add support for multi-layer frame descriptors [1],
> we did not want to try and deduce the frame format, hence this patchset.
> 
> [1] https://github.com/xbmc/xbmc/pull/16102
> 
> Patch 1 adds a new field, format, to the AVDRMFrameDescriptor struct.
> Patch 2-4 adds code to set the new format field.
> 
> Regards,
> Jonas
> 
> ---
> 
> Jonas Karlman (4):
>   hwcontext_drm: Add AVDRMFrameDescriptor.format field
>   hwcontext_vaapi: Set AVDRMFrameDescriptor.format in map_from
>   rkmppdec: Set AVDRMFrameDescriptor.format
>   kmsgrab: Set AVDRMFrameDescriptor.format
> 
>  doc/APIchanges              |  3 +++
>  libavcodec/rkmppdec.c       |  1 +
>  libavdevice/kmsgrab.c       |  1 +
>  libavutil/hwcontext_drm.h   |  4 ++++
>  libavutil/hwcontext_vaapi.c | 38 +++++++++++++++++++++++++++++++++++++
>  libavutil/version.h         |  4 ++--
>  6 files changed, 49 insertions(+), 2 deletions(-)

Can you argue why this case should be put in FFmpeg rather than constructing the format you want in the client code?

The intent of the existing format information is that each layer is definitely usable as the specific format stated if the device supports that format and format modifier.  That isn't true for the top-level format - some devices enforce additional constraints which aren't visible.  For example, if you take an R8 + GR88 frame from an AMD device, it probably won't work as NV12 with Intel video hardware because there the whole frame is required to be in one object (well, not quite - actually the offset from the luma plane to the chroma plane just has some relatively small limit; in practice this gets enforced as single-object, though), but it will work perfectly well as R8 and GR88 planes.

Thanks,

- Mark