[FFmpeg-devel] [PATCH] lavc/amfenc: DXVA2 textures support implementation by AMF encoder

[Mark Thompson]

On 24/04/18 11:51, Alexander Kravchenko wrote:
>> -----Original Message-----
>> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On Behalf Of Mark Thompson
>> Sent: Sunday, April 22, 2018 6:49 PM
>> To: FFmpeg development discussions and patches <ffmpeg-devel at ffmpeg.org>
>> Subject: Re: [FFmpeg-devel] [PATCH] lavc/amfenc: DXVA2 textures support implementation by AMF encoder
>>
>> On 19/04/18 03:18, Song, Ruiling wrote:
>>>> Note that OpenCL <-> D3D11 won't work on AMD for normal video
>>>> surfaces
>>>> (NV12) because there is no support for multiple-plane textures, so
>>>> it's only going to work with DXVA2 currently.  Intel has an extension
>>>> ("cl_intel_d3d11_nv12_media_sharing") which adds a simple hack
>>>> overloading the subresource index and therefore it is usable on Intel
>>>> GPUs, but other vendors don't have that.
>>>
>>> For OpenCL NV12 support, I think we can use two separate images as
>>> arguments, one image for Y plane, and another image for UV plane.
>>> I think AMD OpenCL should support (CL_RG + CL_UNORM_INT8), right?
>>> So, we can get same behavior across different OpenCL vendors.
>>
>> This is exactly what it does already, in a standard way with both DXVA2 and VAAPI - NV12 as R/UNORM_INT8 + RG/UNORM_INT8 is
>> indeed usable for AMD on Windows with DXVA2 interop and via direct upload.
>>
>> The problematic case is D3D11, because the standard cl_khr_d3d11_sharing extension does not support multiple-plane formats.  I
>> would prefer that AMD has an OpenCL-only extension to do it like Intel does, but an alternative route using AMF to do the mapping
>> isn't horrible (though it would be quite confusing if it isn't transparent to the user).
> 
> The AMD OpenCL only extension is requested to be published. Hopefully it is coming soon.

Ah, good!  Will it be the same as the Intel one?  It would be nice if it were defined for any multiplanar format rather than just NV12 (at the moment getting P010 or anything else is only possible in the implementations which share via DRM, currently Beignet and Mali).

> Now it is possible to convert DX11-NV12 surface to OpenCL-NV12 using AMF (AMFSurface->Convert(AMF_MEMORY_OPENCL)).
> The private helper function can be implemented as temporary workaround. It can be deprecated when opencl-only extension is published by AMD

If the extension is arriving any time soon then I think it would be better to wait so that we can just use hwmap like all other cases do.  A via-AMF route is going to be somewhat more inconvenient to combine the required components, and also deprecating public API soon after it is added is not very nice.

- Mark