[FFmpeg-devel] [RFC] amfenc: Add support for OpenCL input

[Mironov, Mikhail]

> -----Original Message-----
> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On Behalf
> Of Mark Thompson
> Sent: January 23, 2018 10:50 AM
> To: ffmpeg-devel at ffmpeg.org
> Subject: Re: [FFmpeg-devel] [RFC] amfenc: Add support for OpenCL input
> 
> On 23/01/18 15:31, Mironov, Mikhail wrote:
> >> -----Original Message-----
> >> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On
> Behalf
> >> Of Mark Thompson
> >> Sent: January 23, 2018 10:21 AM
> >> To: ffmpeg-devel at ffmpeg.org
> >> Subject: Re: [FFmpeg-devel] [RFC] amfenc: Add support for OpenCL
> >> input
> >>
> >> On 23/01/18 15:04, Mironov, Mikhail wrote:
> >>>> -----Original Message-----
> >>>> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On
> >> Behalf
> >>>> Of Mark Thompson
> >>>> Sent: January 22, 2018 6:57 PM
> >>>> To: FFmpeg development discussions and patches <ffmpeg-
> >>>> devel at ffmpeg.org>
> >>>> Subject: [FFmpeg-devel] [RFC] amfenc: Add support for OpenCL input
> >>>>
> >>>> ---
> >>>> This allows passing OpenCL frames to AMF without a download/upload
> >>>> step to get around AMD's lack of support for D3D11 mapping.
> >>>>
> >>>> For example:
> >>>>
> >>>> ./ffmpeg -hwaccel dxva2 -hwaccel_output_format dxva2_vld -i
> >>>> input.mp4 -an -vf
> >>>>
> >>
> 'hwmap=derive_device=opencl,program_opencl=source=examples.cl:kernel=
> >>>> rotate_image' -c:v h264_amf output.mp4
> >>>>
> >>>> * I can't find any documentation or examples for these functions,
> >>>> so I'm guessing a bit exactly how they are meant to work.  In
> >>>> particular, there are some locking functions which I have ignored
> >>>> because I have no idea under what circumstances something might
> >>>> want
> >> to be locked.
> >>>> * I tried to write common parts with D3D11, but I might well have
> >>>> broken
> >>>> D3D11 support in the process - it doesn't work at all for me so I
> >>>> can't test
> >> it.
> >>>> * Not sure how to get non-NV12 to work.  I may be missing
> >>>> something, or it may just not be there - the trace messages suggest
> >>>> it doesn't like the width of
> >>>> RGB0 or the second plane of GRAY8.
> >>>>
> >>>> - Mark
> >>>>
> >>>>
> >>>>  libavcodec/amfenc.c | 178
> >>>> +++++++++++++++++++++++++++++++++++---------
> >>>> --------
> >>>>  libavcodec/amfenc.h |   1 +
> >>>>  2 files changed, 123 insertions(+), 56 deletions(-)
> >>>>
> >>>> ...
> >>>
> >>> AMF encoder works via D3D9 or D3D11 only. AMF OpenCL support is
> done
> >>> for possible integration with external image processing. Passing
> >>> regular
> >> OpenCL 2D images will cause mapping to system memory and copy.
> >>> The fast way is to use interop:
> >>> - Allocate last processing NV12 surface as D3D11 texture
> >>> - iterop it into OpenCL
> >>
> >> This step doesn't work - AMD has no support for mapping NV12 D3D11
> >> textures to OpenCL.  (That's why I wrote the patch at all.)
> >
> > It does via OCL D3D11 Media extension. I use it inside AMF.
> > But for fast results the surface must be allocated with shared handle.
> 
> Where can I find details of that extension?  Currently we use the standard
> cl_khr_d3d11_sharing extension, which combines with Intel's
> cl_intel_d3d11_nv12_media_sharing for NV12 support (see code in
> hwcontext_opencl.c).

This Khronos D3D11 extension:
https://www.khronos.org/registry/OpenCL/sdk/2.0/docs/man/xhtml/clCreateFromD3D11Texture2DKHR.html

The only problem with this extension is that it does not allow to split NV12 into two images: R and RG. I know this this is awkward but you can use AMF for such interop and split. 

> 
> Allocating with shared handle (MISC_SHARED flags) is fine, Intel has that as a
> hard requirement and fails without it.
> 

AMD will not fail but will do additional copies.

Mikhail