[FFmpeg-devel] [PATCH] add phqm filter and img_hash

[Paul B Mahol]

On 10/30/19, Christopher Kennedy <ckennedy at ellation.com> wrote:
> On Wed, Oct 30, 2019 at 10:07 AM Paul B Mahol <onemda at gmail.com> wrote:
>>
>> On 10/30/19, Christopher Kennedy <ckennedy at ellation.com> wrote:
>> > On Sat, Oct 26, 2019 at 9:15 AM Paul B Mahol <onemda at gmail.com> wrote:
>> >>
>> >> On 10/26/19, Christopher Kennedy <ckennedy at ellation.com> wrote:
>> >> > On Sat, Oct 26, 2019 at 8:22 AM Paul B Mahol <onemda at gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> On 10/26/19, Christopher Kennedy <ckennedy at ellation.com> wrote:
>> >> >> > This is a reference/encode comparison filter with two files input
>> >> >> > like
>> >> >> > the psnr or vmaf filter.
>> >> >> > So it is completely different and uses the C++ OpenCV API since
>> >> >> > this
>> >> >> > img_hash library is not in the C API.
>> >> >> > It's unique to what the OCV filter does, and has more research
>> >> >> > implications from my talk at Demuxed 2019.
>> >> >>
>> >> >> I do not see how that is relevant.
>> >> >>
>> >> >> There should be generic opencv filter which could do this above in
>> >> >> generic way, and not by adding yet another filter that uses only
>> >> >> some
>> >> >> part of opencv.
>> >> >
>> >> > Is it really possible to do framesync() operations like dual input
>> >> > filters
>> >> > like psnr/vmaf and also handle input/output rendering of the frames
>> >> > too?
>> >> > This sounds odd to me but I would love to understand how this is
>> >> > possible.
>> >>
>> >> framesync is nothing special, its just used a lot, there are video
>> >> filters that do not use it and still operate on multiple inputs.
>> >>
>> >> >
>> >> > The C OpenCV API is not recommended so this does the OpenCV part
>> >> > in C++ which allows it to be fully utilized and supported. So that
>> >> > seems
>> >> > better to me than using the API OpenCV won't really support and
>> >> > doesn't
>> >> > allow usage of img_hash (it is NOT in the C API of OpenCV,
>> >> > impossible
>> >> > to
>> >> > use).
>> >> >
>> >> > The OpenCV C++ img_hash library is the fastest implementation and
>> >> > does
>> >> > work best in OpenCV. So implementing this in C directly isn't a task
>> >> > I
>> >> > believe
>> >> > is good to do.
>> >>
>> >> I do not care how generic opencv filter is done, it can be C++ just
>> >> fine.
>> >>
>> >> >
>> >> > So should the current OpenCV stuff be merged into this filter, is
>> >> > that
>> >> > possible?
>> >>
>> >> Current opencv stuff in libavfilter is pretty dead and not maintained
>> >> at
>> >> all.
>> >> So I'm not really fond of adding yet another opencv filter that not
>> >> gonna be maintained.
>> >
>> > Well I would maintain it, but maybe I should instead run a Fork of
>> > FFmpeg
>> > with
>> > C++ and OpenCV properly done.
>> >
>> > Yes the current OCV stuff is dead and not supported by OpenCV or
>> > maintained
>> > it sounds like.
>> >
>> > So if we can't put C++ into FFmpeg (yet we already have with decklink
>> > filters,
>> > so that is unfair to say).
>> >
>> > And we can't switch to use OpenCV C++ yet the C API is dead and not
>> > supported.
>>
>> Who said that?
>> C++ module filter in libavfilter dealing with latest OpenCV and well
>> maintained is always welcome.
>
>
> So I should be moving the functionality of the current ocv filter into a
> C++ file for filters to use like phqm, the current ocv transforms?
>
> Is this a good plan, separate the C++ from the vf_ filter as I have done
> with img_hash.cpp (yet make it a more generic set of OpenCV C++
> helper functions to plug into C vf_filters which can be separated by
> function yet unified in code used from the C++ base code)?

IMHO filter file can be cpp file without issues. And even if not use
ff_ prefixed functions,

>
> Wanting to clarify exactly what is acceptable so I can do it and not waste
> time / effort doing stuff that isn't right.
>
> Thanks,
> Christopher
>
>
>>
>> >
>> > The only solution I see is to run a fork which is sad to have to do.
>> >
>> > This just seems really unreasonable overall :(.
>> >
>> > Thanks,
>> > Christopher.
>> >
>> >
>> >>
>> >> > If so, then your saying the PSNR filter could also alter frames and
>> >> > output
>> >> > them
>> >> > too from the reference/encode or one or the other? I need to
>> >> > understand
>> >> > this
>> >> > better, I want to make it right so am listening.
>> >>
>> >> I'm not sure what you really want. Native psnr filter does not alter
>> >> video,
>> >> it just takes two inputs and output first one unmodified and reports
>> >> via log and metadata psnr values it measured.
>> >>
>> >> Also, I'm of opinion that hashing images is so trivial operation that
>> >> could be done native to libavfilter.
>> >>
>> >> >
>> >> > Thanks
>> >> > Christopher
>> >> >
>> >> >>
>> >> >> >
>> >> >> > Christopher
>> >> >> >
>> >> >> > On Sat, Oct 26, 2019 at 7:38 AM Paul B Mahol <onemda at gmail.com>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> Why is this not generic filter like already existing opencv
>> >> >> >> filter?
>> >> >> >>
>> >> >> >> On 10/26/19, ckennedy at ellation.com <ckennedy at ellation.com>
>> >> >> >> wrote:
>> >> >> >> > From: Christopher Kennedy <ckennedy at ellation.com>
>> >> >> >> >
>> >> >> >> > this adds a phqm filter and OpenCV img_hash based resource
>> >> >> >> > usable
>> >> >> >> > by the phqm and future filters using image hash functionality
>> >> >> >> > from OpenCV.
>> >> >> >> >
>> >> >> >> > C++ to C handling so that full OpenCV functionality and API
>> >> >> >> > can
>> >> >> >> > be used instead of the C versions (which are incomplete and
>> >> >> >> > don't always exist).
>> >> >> >> >
>> >> >> >> > Example command line:
>> >> >> >> >
>> >> >> >> > ffmpeg -i encode.mp4 -i reference.mp4 \
>> >> >> >> >            -filter_complex "[0:v][1:v]phqm=stats_file=out.log"
>> >> >> >> > \
>> >> >> >> >            -y -f null /dev/null
>> >> >> >> >
>> >> >> >> > Signed-off-by: Christopher Kennedy <ckennedy at ellation.com>
>> >> >> >> >
>> >> >> >> > _______________________________________________
>> >> >> >> > ffmpeg-devel mailing list
>> >> >> >> > ffmpeg-devel at ffmpeg.org
>> >> >> >> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> >> >> >> >
>> >> >> >> > To unsubscribe, visit link above, or email
>> >> >> >> > ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>> >> >> >
>> >> >
>> >
>