[FFmpeg-devel] [PATCH v2] avfilter: add libdewobble_opencl filter

[Daniel Playfair Cal]

On Tue, Aug 24, 2021 at 11:42 PM Mapul Bhola
<ffmpegandmahanstreamer at e.email> wrote:

> I agree with Mahol here. It's good to make sure all the code in FFmpeg meets a certain quality.
> I thought there were OpenCV filters in ffmpeg already?


I'm more than happy to address any issues of code quality in this
filter to go into FFmpeg - please let me know if you can see something
specific that I haven't addressed.

I don't think using OpenCV is a quality issue. OpenCV is an
established and well maintained library, and the specific algorithms
used from it in Dewobble are well used and tested. Within Dewobble
there are some functionalities that are implemented natively and
others that use OpenCV implementations. My choices have depended on
how much specialisation is needed and the relative difficulty of
implementing the algorithms. For example, I wrote my own OpenCL
kernels to build the final map for warping and for colour conversion.
But I didn't write my own implementation of warping/interpolation,
Shi-Tomasi corner detection, Lucas-Kanade optical flow, RANSAC, etc.
There is no point IMO making such a huge effort to recreate what is
already there in OpenCV, unless there is a good reason to think the
result would be better. There are also alternative algorithms for
video stabilization, many of which are also implemented in OpenCV. So
it's easy to experiment with different methods without having to
implement complex computer vision algorithms each time.

For those parts that don't change so much, which are customised more,
or which could be done better than by using OpenCV, I will probably
slowly implement algorithms in Dewobble (and any help is welcome).

And yes, there is an existing filter "ocv" (vf_libopencv.c) which
wraps a very specific set of functionalities in OpenCV, from its image
filtering category. These functionalities are unrelated to this filter
or to Dewobble. There is also a filter "deshake_opencl" which doesn't
depend on OpenCV but contains code copied from some of it's OpenCL
kernels.

> And if you are the writer of this plugin as well, you should consider relicensing it to LGPL for its usage in ffmpeg.

I considered this but I've decided to license Dewobble as GPL. I
realise this prevents it from being used in closed source or
permissively licensed libraries that links to it, but it can still be
usable for end users of FFmpeg and within other GPL software. FFmpeg
already has build infrastructure to support this, as well as multiple
existing filters which are also licensed under GPL.