[FFmpeg-devel] [PATCH] add phqm filter and img_hash

[Christopher Kennedy]

On Wed, Oct 30, 2019 at 10:57 AM Paul B Mahol <onemda at gmail.com> wrote:
>
> On 10/30/19, Christopher Kennedy <ckennedy at ellation.com> wrote:
> > On Wed, Oct 30, 2019 at 10:07 AM Paul B Mahol <onemda at gmail.com> wrote:
> >>
> >> On 10/30/19, Christopher Kennedy <ckennedy at ellation.com> wrote:
> >> > On Sat, Oct 26, 2019 at 9:15 AM Paul B Mahol <onemda at gmail.com> wrote:
> >> >>
> >> >> On 10/26/19, Christopher Kennedy <ckennedy at ellation.com> wrote:
> >> >> > On Sat, Oct 26, 2019 at 8:22 AM Paul B Mahol <onemda at gmail.com>
> >> >> > wrote:
> >> >> >>
> >> >> >> On 10/26/19, Christopher Kennedy <ckennedy at ellation.com> wrote:
> >> >> >> > This is a reference/encode comparison filter with two files input
> >> >> >> > like
> >> >> >> > the psnr or vmaf filter.
> >> >> >> > So it is completely different and uses the C++ OpenCV API since
> >> >> >> > this
> >> >> >> > img_hash library is not in the C API.
> >> >> >> > It's unique to what the OCV filter does, and has more research
> >> >> >> > implications from my talk at Demuxed 2019.
> >> >> >>
> >> >> >> I do not see how that is relevant.
> >> >> >>
> >> >> >> There should be generic opencv filter which could do this above in
> >> >> >> generic way, and not by adding yet another filter that uses only
> >> >> >> some
> >> >> >> part of opencv.
> >> >> >
> >> >> > Is it really possible to do framesync() operations like dual input
> >> >> > filters
> >> >> > like psnr/vmaf and also handle input/output rendering of the frames
> >> >> > too?
> >> >> > This sounds odd to me but I would love to understand how this is
> >> >> > possible.
> >> >>
> >> >> framesync is nothing special, its just used a lot, there are video
> >> >> filters that do not use it and still operate on multiple inputs.
> >> >>
> >> >> >
> >> >> > The C OpenCV API is not recommended so this does the OpenCV part
> >> >> > in C++ which allows it to be fully utilized and supported. So that
> >> >> > seems
> >> >> > better to me than using the API OpenCV won't really support and
> >> >> > doesn't
> >> >> > allow usage of img_hash (it is NOT in the C API of OpenCV,
> >> >> > impossible
> >> >> > to
> >> >> > use).
> >> >> >
> >> >> > The OpenCV C++ img_hash library is the fastest implementation and
> >> >> > does
> >> >> > work best in OpenCV. So implementing this in C directly isn't a task
> >> >> > I
> >> >> > believe
> >> >> > is good to do.
> >> >>
> >> >> I do not care how generic opencv filter is done, it can be C++ just
> >> >> fine.
> >> >>
> >> >> >
> >> >> > So should the current OpenCV stuff be merged into this filter, is
> >> >> > that
> >> >> > possible?
> >> >>
> >> >> Current opencv stuff in libavfilter is pretty dead and not maintained
> >> >> at
> >> >> all.
> >> >> So I'm not really fond of adding yet another opencv filter that not
> >> >> gonna be maintained.
> >> >
> >> > Well I would maintain it, but maybe I should instead run a Fork of
> >> > FFmpeg
> >> > with
> >> > C++ and OpenCV properly done.
> >> >
> >> > Yes the current OCV stuff is dead and not supported by OpenCV or
> >> > maintained
> >> > it sounds like.
> >> >
> >> > So if we can't put C++ into FFmpeg (yet we already have with decklink
> >> > filters,
> >> > so that is unfair to say).
> >> >
> >> > And we can't switch to use OpenCV C++ yet the C API is dead and not
> >> > supported.
> >>
> >> Who said that?
> >> C++ module filter in libavfilter dealing with latest OpenCV and well
> >> maintained is always welcome.
> >
> >
> > So I should be moving the functionality of the current ocv filter into a
> > C++ file for filters to use like phqm, the current ocv transforms?
> >
> > Is this a good plan, separate the C++ from the vf_ filter as I have done
> > with img_hash.cpp (yet make it a more generic set of OpenCV C++
> > helper functions to plug into C vf_filters which can be separated by
> > function yet unified in code used from the C++ base code)?
>
> IMHO filter file can be cpp file without issues. And even if not use
> ff_ prefixed functions,

Okay how about this layout...

libavfilter/opencv.cpp (all OpenCV code, using C++ API, helper
functions for use by the filters)

libafilter/vf_libopencv.c   current filter, but uses opencv.cpp
functions for OpenCV code

libavfilter/vf_phqm.c       uses OpenCV functions from opencv.cpp

Since OpenCV ranges in varying functions we may need, and doing
it as C++ cleanly separately in one place sharing with the C filters seems nice.

I agree the duplication of OpenCV in C and C++ APIs seems bad, and
using just the C++
API is what I see as best according to the OpenCV devs statements
against using the C API.


Is that acceptable or is a better layout suggested?

Thanks,
Christopher

>
> >
> > Wanting to clarify exactly what is acceptable so I can do it and not waste
> > time / effort doing stuff that isn't right.
> >

> >> >
> >