[FFmpeg-devel] [PATCH 1/2] libavutil/libavfilter: opencl wrapper based on comments on 20130327

[Wei Gao]

HI,

Stefano Sabatini, thanks for your reply, questions and explanations

Thanks
Best regards

2013/3/28 Stefano Sabatini <stefasab at gmail.com>

> On date Wednesday 2013-03-27 21:21:09 +0800, Wei Gao encoded:
> >
>
> > From 544da77d67c6c27e415363c3ebd2f1894f98932e Mon Sep 17 00:00:00 2001
> > From: highgod0401 <highgod0401 at gmail.com>
> > Date: Wed, 27 Mar 2013 21:17:29 +0800
> > Subject: [PATCH] opencl wrapper based on comments on 20130327
> >
> > +
> > +typedef struct AVOpenCLExternalInfo {
> > +    cl_platform_id platform;
> > +    cl_device_type device_type;
> > +    cl_context context;
> > +    cl_device_id *device_ids;
> > +    cl_device_id  device_id;
> > +    cl_command_queue command_queue;
> > +    char *platform_name;
> > +} AVOpenCLExternalInfo;
>
> About the overall design. It is pretty difficult for me to provide a
> meaningful design since I have no serious knowledge about the OpenCL
> architecture, but I'll try to sketch something which could match the
> FFmpeg architecture and conventions.
>
> Let's start with something simple, which we can extend later.
>
> First of all, you need a context for working with OpenCL related
> structures, let's call it AVOpenCLContext, which you could create for
> example with:
>
> int av_opencl_open(AVOpenCLContext **ctx, AVDictionary *options);
>
> you set the options, and the function allocates the context and fills
> it with the required fields.
>
> Alternatively, suppose you want to reuse some structures already
> created, you could have:
> int av_opencl_open(AVOpenCLContext *ctx, AVDictionary *options);
>
> in this case you create and fill the context before opening it with
> the function (this could be useful to pass pointers to binary
> structures).
>
> At this point you need to register a kernel:
>
> int av_opencl_register_kernel(AVOpenCLContext *ctx,
>                               const char *kernel_name, const char
> *kernel_code);
>
> I wonder if it makes sense to create a program *per kernel* and
> compile the program during kernel registration.
>
> think about that if on kernel is in use and a nother filter is registered,
the program will stop and build all kernels,so the runing kenrel may be
stop until finish the following kernel compile. if we compile all the
kernel on time, the application just use it and will not happen this case.
the application just use it.

Alternatively you could bookkeep several programs, and specify the
> program where you want to register the kernel, for example:
>
> int av_opencl_register_kernel(AVOpenCLContext *ctx,
>                               const char *program_name,
>                               const char *kernel_name,
>                               const char *kernel_code);
>
no int av_opencl_register_kernel(const char *kernel_entry_point, const char
*kernel_code); this is in the filter register function. it can record all
the kernel code and save it, and the kernel code will be compiled in the
av_opencl_init(), first record all kernel code, secode init the opencl
enviroment. so you can init once, and reuse the resource.

>
> then you could have a function which does:
> av_opencl_compile_program(AVOpenCLContext *ctx, const char *program_name);
>
> which compiles the program with the associated kernels (I don't know
> which are the drawbacks of this approach, and if it is feasible for
> your use case).
>
this is in the av_opencl_init() function, and it will compile once for all
kernel, so all other kernels can not be compiled after call
 av_opencl_init()  once. I think it should not be a api, because kernels
should be compiled at the init step of program, and the application should
just use the compiled kernel, not to compiled when use it. if the
application compile in runtime, it will call the compile api for a many
times.because the application should know all the kernels it have. also we
will add the binary file operation that the kernel will not be compiled in
in the next calling time. the kernel just compiled for once.

>
> At this point you need a function to create a kernel handle associated
> to a kernel (possibly already compiled in a given program).
>
> This could be:
> int av_opencl_create_kernel(AVOpenCLContext *ctx,
>                             const char *program_name,
>                             const char *kernel_name,
>                             const char *kernel_entry_point,
>                             av_opencl_kernel_function function);
>
> which has also the advantage that you specify the kernel function, so
> you don't need a separate registration function for that.
>
one function may be run many kernels(such the scale kernel I have implement
in our test version)

>
> Then you finally do:
> int av_opencl_run_kernel(AVOpenCLContext *ctx,
>                          const char *program_name, const char *kernel_name,
>                          void **userdata)
>
> which executes the function associated to a program and a kernel, by
> passing the data in userdata.
>
> When you are done with a program (and all the related kernels), you do:
> av_opencl_destroy_program(AVOpenCLContext *ctx, const char *program_name);
>
> Alternatively, we could have a single program per AVOpenCLContext, if
> this seems overkill.

Same for the buffer API, all the related functions could reference a
> specific AVOpenCL context.
>
> To destroy the context, you finally do:
> void av_opencl_close(AVOpenCLContext *ctx);
>
> which also cleans up all the associated structures.
>
> ...
>
> The good thing with this approach is that you don't need a global per
> application context (which is usually brittle or just broken by
> design, especially for a complex library collection like FFmpeg), and
> you can eventually share the same context amongst several components.
>
we discessed with michaelni, and checked OpenCL Specification Version: 1.2,
the context ,command queue, program is thread safe, so I think we can use
on for saving resource

>
> Do you think that such design could fit your use case?
>
> Could it be actually implemented within the OpenCL framework?
>
> We can extend this design in order to accomodate your specific use
> cases.
>
> [...]
> --
> FFmpeg = Fanciful & Foolish Mystic Political Empowered Gnome
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>