[FFmpeg-devel] lavfi state of affairs 3

[Stefano Sabatini]

On date Wednesday 2009-07-01 12:06:48 -0700, Baptiste Coudurier encoded:
> Hi Stefano,
> 
> Stefano Sabatini wrote:
[...]
> > I'll try to resume here the main current issues.
> > 
> > |* Expand filter missing.
> > |
> > |  There is the need for an expand filter with "direct rendering", that
> > |  is which doens't require memcpy of any sort, this maybe requires API
> > |  extension:
> > |  http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/85015
> 
> Is this mandatory ? Direct rendering is not mandatory IMHO.

Michael explicitely asked for that.
 
> > |  MPlayer already has such a thing, so it may be worthwhile check that
> > |  code, in general is a good idea to check out all the filters in
> > |  MPlayer/libmpcodecs in order to avoid to waste time reinventing the
> > |  wheel.
> 
> What about the license ? Is it GPL ? If so I would prefer a LGPL
> implementation.

I think most filters in libmpcodecs are GPL, Michael explicitely said
I can relicense his own code as LGPL if required.

Anyway libmpcodecs and libavfilter are different enough so that it is
not in general possible to do a mechanical rewrite of a filter, also
note the pad filter I wrote is already memcpy-less.

As for the specific case of the direct rendering what is left is to
understand how to use the FFmpeg libavcodec get/release_buffer(), I
still didn't digged into it so I can't really say how hard that would
be.

> > |  Same considerations apply for the syntax, when possible is a good
> > |  idea to keep the same syntax (when sane) of the corresponding
> > |  MPlayer filters.
> > |
> > |  An implementation of an expand/pad filters has been already posted
> > |  on the soc ML:
> > |  http://thread.gmane.org/gmane.comp.video.ffmpeg.soc/2779/
> > 
> > I worked out a framework for argument passing to filters, and I
> > currently have a working implementation for such a pad filter:
> > http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/89945
> > 
> > The patch proposes a change in how the buffers are requested, they are
> > requested to the terminating filter, and allows direct rendering but
> > still doesn't implement it. Ideally the decoder should request a
> > buffer from the chain and write to it, rather than memcpy the buffered
> > frame to the buffer provided by the filterchain.
> > 
> > So the thread revealed this other two missing points:
> > 
> > * missing direct rendering
> 
> IMHO not mandatory if it can work without.
> 
> > * need for a regression test covering some of the most common use case
> >   (cropping + padding + scaling). I posted an embryonal regression
> >   test for lavfi here:
> >   http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/92792 
> 
> Well, good and wanted but strictly not mandatory to get it into svn.

But this shouldn't be really hard to implement, just we need to figure
out *which* tests we should perform, the framework is already there,
then it should be easier to address the other points.
 
> >   which is more a proof of concept, than I made some changes to the
> >   main repo regression tests and wanted to update the lavfi repo, and
> >   got stucked there.
> > 

[...]
> > |* A more powerful system for dealing with image formats is needed to
> > |  avoid bad code duplication. I mean some system to tell if a format
> > |  is for example packed, planar, the number of channels/planes and so
> > |  on, as discussed here:
> > |  http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/83733
> > |
> > |  and creates corresponding AVFilterFormats lists.
> > |
> > |  pixdesc.[ch] already committed in SVN may provide the foundations
> > |  for such a system (I intend to work on this).
> > 
> > The foundations of such a system (pixdesc.{h,c}) is already committed
> > in the main repo, altough disabled, some nice features of it (for
> > example av_get_bits_per_pixel()) should simplify writing filters.
> 
> Is it needed to have equal functionality as what is in svn right now ?

No, and I don't consider it mandatory too, mentioned it just because
it was mentioned in the original message.
 
> > |* Maybe a one-pixel colorspace transform, as discussed here:
> > |  http://thread.gmane.org/gmane.comp.video.ffmpeg.soc/5727/focus=5739
> > |
> > |  may avoid code duplication for all that filters that need to execute
> > |  YUV <-> RGB or some other transformation just for one color.
> > |
> > |  I'm not sure what's the better place for it, but lsws seems the
> > |  obvious choice.
> > 
> > There is some code for that already committed in
> > libavfilter/parseutils.c, but I don't believe this feature is anyway
> > blocking, after all whenever I needed to implement such a one pixel
> > transform I used the macros in libavcodec/colorspace.h, also the
> > libswscale soc taks may help here.
> 
> How is that mandatory ?

Maybe nothing at all, same as above.

[...]
> > Another important point is:
> > 
> > * missing variable frame size/pixel format support, that should be
> >   managed by lavfi, currently this feature is implemented in ffmpeg.c,
> >   I posted a very rudimental patch for that much time ago:
> >   http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/78477
> > 
> > Any help is very appreciated, libavfi looks like an impressive
> > addition to FFmpeg, also it should allow to significantly reduce the
> > clutter in the apps.
> 
> Definitely, but we need to focus on what's mandatory for svn inclusion,
> not what we would like to have.
> 
> It's been 4 months vhook has been removed now, and we do not have a
> replacement yet, and people starts to work on gsoc code because they
> have to.
> 
> What we need IMHO is:
> 1) sane API to permit work on other filters.
> 
> How is the API ? What is missing ? AFAIK I saw some people working ith
> it so it must be working in some ways.

As long as I know the API is almost feature complete, but for the
missing variable frame size/pixel support.

Also check this:
http://thread.gmane.org/gmane.comp.video.ffmpeg.devel/92130

> 2) crop replacement
> 
> Do we have a working filter for it ?

Yes. IIRC this filter results broken after my get_buffer() API change
proposal required for the direct rendering, fixing it is one of the
missing points for getting that patch acceptable (also it shows how a
regression test may help there).

> 3) scale replacement with support for changing resolution.
> 
> Do we have a working filter for it ?

scale/API needs to support variable frame size/pixel format.

Maybe this may work making every source of the chain notify and
reconfigure the chain when an frame size/pixel format change is
detected.

Regards.
-- 
FFmpeg = Friendly and Fierce Magic Programmable Enlightening Ghost