[FFmpeg-devel] [PATCH] libavfilter: temporarily remove DNN framework and vf_sr filter

[Ronald S. Bultje]

Hi,

On Fri, Jul 27, 2018 at 10:59 AM, Pedro Arthur <bygrandao at gmail.com> wrote:

> 2018-07-27 11:31 GMT-03:00 Ronald S. Bultje <rsbultje at gmail.com>:
> > Hi,
> >
> > On Fri, Jul 27, 2018 at 10:16 AM, Pedro Arthur <bygrandao at gmail.com>
> wrote:
> >
> >> Hi,
> >>
> >> 2018-07-27 8:12 GMT-03:00 Ronald S. Bultje <rsbultje at gmail.com>:
> >> >
> >> > NN is different. The exposed software *uses* but does not *generate*
> the
> >> > coefficients. In fact, the meaning of most coefficients is completely
> >> > opaque. They are just multiplier and addition constants with no
> obvious
> >> > relationship to reality, other than that they were generated by some
> >> > magical process that tells us that statistically (??), this bears
> >> > resemblence to some trained form of reality. You want to adjust
> behaviour
> >> > for a defined different outcome? You'll have to retrain. That's all
> fine,
> >> > don't get me wrong, but we need to know how to retrain since it's not
> >> > obvious to us. Just tell us what process and what data were used to
> get
> >> the
> >> > outcome so that we can get a different outcome if we so choose (that
> *is*
> >> > freedom to modify)?
> >> >
> >> > Please document (and make available) all such information for the
> >> > re-training.
> >> We already did it [1] the repo provides all scripts to download the
> >> data and run the training and also generates the header with weights
> >> readily to use in ffmpeg.
> >> When Jean-Baptist raised this issue we worked to provide all the
> >> necessary scripts to perform training, reading the replies above, it
> >> seems no one is  aware of it (not even J-B).
> >> Isn't [1] enough? whats is missing then? why no one asked for whatever
> >> may be missing in the original email when we provided the link to the
> >> repo?
> >>
> >>
> >> [1] - https://github.com/HighVoltageRocknRoll/sr
> >
> >
> > It's probably because the documentation is not in FFmpeg, so how could we
> > be aware of it?
> >
> > $ git grep -i rocknroll .
> > $ pwd
> > /Users/ronaldbultje/Projects/ffmpeg
> > $ git log --oneline -1
> > 536bcc3 avcodec: add missing files missed in previous commits
> >
> > It sounds to me like we agree that documentation for re-training should
> be
> > provided, and the miscommunication was more about where this should be
> > and/or how to make it discoverable. I would encourage you to make it as
> > discoverable as possible by including the documentation inside the FFmpeg
> > source tree, possibly inside the filter's source code file.
> >
> > (I understand that it may not be practical for the actual retraining data
> > and scripts to be hosted on videolan/FFmpeg infrastructure.)
> A small description with a link for the repo in the header with the
> weights is enough?


If it answers the questions (after the obvious "hey, that looks cool, let
me dig deeper") "what on earth is this" and/or "how can I do that myself?",
then maybe, yes? It might also be useful to briefly refer to (or repeat)
the same in public documentation, e.g. in filters.texi, so that users who
start reading about nn filters and want to pass their own coefficients can
read easily how to generate them. Freedoms aren't just intended for
developers, they're also meant for users.

Ronald