[FFmpeg-devel] [PATCH] add phqm filter and img_hash

[Christopher Kennedy]

On Wed, Oct 30, 2019 at 10:07 AM Paul B Mahol <onemda at gmail.com> wrote:
>
> On 10/30/19, Christopher Kennedy <ckennedy at ellation.com> wrote:
> > On Sat, Oct 26, 2019 at 9:15 AM Paul B Mahol <onemda at gmail.com> wrote:
> >>
> >> On 10/26/19, Christopher Kennedy <ckennedy at ellation.com> wrote:
> >> > On Sat, Oct 26, 2019 at 8:22 AM Paul B Mahol <onemda at gmail.com> wrote:
> >> >>
> >> >> On 10/26/19, Christopher Kennedy <ckennedy at ellation.com> wrote:
> >> >> > This is a reference/encode comparison filter with two files input
> >> >> > like
> >> >> > the psnr or vmaf filter.
> >> >> > So it is completely different and uses the C++ OpenCV API since this
> >> >> > img_hash library is not in the C API.
> >> >> > It's unique to what the OCV filter does, and has more research
> >> >> > implications from my talk at Demuxed 2019.
> >> >>
> >> >> I do not see how that is relevant.
> >> >>
> >> >> There should be generic opencv filter which could do this above in
> >> >> generic way, and not by adding yet another filter that uses only some
> >> >> part of opencv.
> >> >
> >> > Is it really possible to do framesync() operations like dual input
> >> > filters
> >> > like psnr/vmaf and also handle input/output rendering of the frames
> >> > too?
> >> > This sounds odd to me but I would love to understand how this is
> >> > possible.
> >>
> >> framesync is nothing special, its just used a lot, there are video
> >> filters that do not use it and still operate on multiple inputs.
> >>
> >> >
> >> > The C OpenCV API is not recommended so this does the OpenCV part
> >> > in C++ which allows it to be fully utilized and supported. So that
> >> > seems
> >> > better to me than using the API OpenCV won't really support and doesn't
> >> > allow usage of img_hash (it is NOT in the C API of OpenCV, impossible
> >> > to
> >> > use).
> >> >
> >> > The OpenCV C++ img_hash library is the fastest implementation and does
> >> > work best in OpenCV. So implementing this in C directly isn't a task I
> >> > believe
> >> > is good to do.
> >>
> >> I do not care how generic opencv filter is done, it can be C++ just fine.
> >>
> >> >
> >> > So should the current OpenCV stuff be merged into this filter, is that
> >> > possible?
> >>
> >> Current opencv stuff in libavfilter is pretty dead and not maintained at
> >> all.
> >> So I'm not really fond of adding yet another opencv filter that not
> >> gonna be maintained.
> >
> > Well I would maintain it, but maybe I should instead run a Fork of FFmpeg
> > with
> > C++ and OpenCV properly done.
> >
> > Yes the current OCV stuff is dead and not supported by OpenCV or maintained
> > it sounds like.
> >
> > So if we can't put C++ into FFmpeg (yet we already have with decklink
> > filters,
> > so that is unfair to say).
> >
> > And we can't switch to use OpenCV C++ yet the C API is dead and not
> > supported.
>
> Who said that?
> C++ module filter in libavfilter dealing with latest OpenCV and well
> maintained is always welcome.


So I should be moving the functionality of the current ocv filter into a
C++ file for filters to use like phqm, the current ocv transforms?

Is this a good plan, separate the C++ from the vf_ filter as I have done
with img_hash.cpp (yet make it a more generic set of OpenCV C++
helper functions to plug into C vf_filters which can be separated by
function yet unified in code used from the C++ base code)?

Wanting to clarify exactly what is acceptable so I can do it and not waste
time / effort doing stuff that isn't right.

Thanks,
Christopher


>
> >
> > The only solution I see is to run a fork which is sad to have to do.
> >
> > This just seems really unreasonable overall :(.
> >
> > Thanks,
> > Christopher.
> >
> >
> >>
> >> > If so, then your saying the PSNR filter could also alter frames and
> >> > output
> >> > them
> >> > too from the reference/encode or one or the other? I need to understand
> >> > this
> >> > better, I want to make it right so am listening.
> >>
> >> I'm not sure what you really want. Native psnr filter does not alter
> >> video,
> >> it just takes two inputs and output first one unmodified and reports
> >> via log and metadata psnr values it measured.
> >>
> >> Also, I'm of opinion that hashing images is so trivial operation that
> >> could be done native to libavfilter.
> >>
> >> >
> >> > Thanks
> >> > Christopher
> >> >
> >> >>
> >> >> >
> >> >> > Christopher
> >> >> >
> >> >> > On Sat, Oct 26, 2019 at 7:38 AM Paul B Mahol <onemda at gmail.com>
> >> >> > wrote:
> >> >> >>
> >> >> >> Why is this not generic filter like already existing opencv filter?
> >> >> >>
> >> >> >> On 10/26/19, ckennedy at ellation.com <ckennedy at ellation.com> wrote:
> >> >> >> > From: Christopher Kennedy <ckennedy at ellation.com>
> >> >> >> >
> >> >> >> > this adds a phqm filter and OpenCV img_hash based resource usable
> >> >> >> > by the phqm and future filters using image hash functionality
> >> >> >> > from OpenCV.
> >> >> >> >
> >> >> >> > C++ to C handling so that full OpenCV functionality and API can
> >> >> >> > be used instead of the C versions (which are incomplete and
> >> >> >> > don't always exist).
> >> >> >> >
> >> >> >> > Example command line:
> >> >> >> >
> >> >> >> > ffmpeg -i encode.mp4 -i reference.mp4 \
> >> >> >> >            -filter_complex "[0:v][1:v]phqm=stats_file=out.log" \
> >> >> >> >            -y -f null /dev/null
> >> >> >> >
> >> >> >> > Signed-off-by: Christopher Kennedy <ckennedy at ellation.com>
> >> >> >> >
> >> >> >> > _______________________________________________
> >> >> >> > ffmpeg-devel mailing list
> >> >> >> > ffmpeg-devel at ffmpeg.org
> >> >> >> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >> >> >> >
> >> >> >> > To unsubscribe, visit link above, or email
> >> >> >> > ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
> >> >> >
> >> >
> >