[FFmpeg-devel] [PATCH] libavfilter: Add more operation supports in FFmpeg dnn native mode.

[Pedro Arthur]

Em seg, 29 de abr de 2019 às 00:06, <xwmeng at pku.edu.cn> escreveu:
>
>
>
>
> > -----原始邮件-----
> > 发件人: "Pedro Arthur" <bygrandao at gmail.com>
> > 发送时间: 2019-04-29 10:42:42 (星期一)
> > 收件人: "FFmpeg development discussions and patches" <ffmpeg-devel at ffmpeg.org>
> > 抄送:
> > 主题: Re: [FFmpeg-devel] [PATCH] libavfilter: Add more operation supports in FFmpeg dnn native mode.
> >
> > I think for training these filters the preferred method is VALID as it
> > uses only the data available (without filling the borders) and gives
> > the best possible result.
> > However for inference usually one expects to output an image with the
> > same size of the original (imagine the case of chained filters where
> > each one reduces the image by a few pixels, in the end one may have a
> > useless output).
> > Therefore it makes perfect sense to use different padding methods for
> > training/inference.
> >
> > The clamp_to_edge padding was introduced before the TF backend thus it
> > stayed in the native backend even after the introduction of the TF
> > backend.
> > Indeed the clamp_to_edge is simpler than the other padding methods and
> > also gives a slight better result, If I remember correct the student
> > which implemented the TF backend did not find an equivalent padding
> > method in TF, thats why it uses different paddings.
> >
> Yes, I think clamp_to_edge is a good method to keep the output with the same size as input. However, I don't think "VALID" is the best method giving best possible result. So, for "VALID" mode, maybe we can use the clamp_to_edge method in the current dnn native mode? And then, we should also add "SAME" option to support other filters.
>

I think it is best to not make any assumptions like VALID =>
clamp_to_edge, but you can keep it for now.
Ideally the model should have a padding layer which the backend
properly implements. Currently the TF backend when reading a native
model adds this padding layer implicitly, therefore it would be a
matter of changing it to have an explicity padding layer in the model.

Maybe you can assume VALID => clamp_to_edge, so you can add what you
need without changing the SR code and later you implement the
explicity padding support and send a PR to the original repo
(https://github.com/HighVoltageRocknRoll/sr) properly modifying the
model.