[FFmpeg-devel] [PATCH] [2/??] [3/3] Filter graphs - Parser for a graph description

[Aurelien Jacobs]

Michael Niedermayer wrote:

> On Tue, Mar 18, 2008 at 11:13:25PM +0100, Aurelien Jacobs wrote:
> > Michael Niedermayer wrote:
> > 
> > > On Tue, Mar 18, 2008 at 09:29:51AM +0000, vmrsss wrote:
> > > [...]
> > > > (F) If I understood your examples correctly (which I might have
> > > > not)`, there is still one missing ingredient: a feedback
> > > > combinator, say !, which takes a filter with at least one input
> > > > and at least one output stream and feeds back the first output
> > > > to the first input. For instance, let's look at the examples
> > > > you have been discussing in this thread:
> > > > 
> > > > (1)
> > > > > in --> scale --> crop --> picInPic --> rotate --> split -->
> > > > > out ^                      |
> > > > >                             |                      |
> > > > >                           delay<-------------------/
> > > > 
> > > > 
> > > > becomes: scale,crop, !( picInPic, rotate, split, delay * nop )
> > > > -- in fact the duplicated stream from split is fed to delay and
> > > > delay's output is then fed back to picInPic (if that was the
> > > > sense of the example, of course...)
> > > 
> > > How does the following work in your system?
> > > 
> > > (in0,tmp0,tmp2)filter1(tmp1,out0,tmp2);(in1,tmp1,tmp3)filter2(tmp3,tmp0,out1)
> > 
> > If I understood correctly, this would look like this:
> > 
> > !(swap*nop,
> >      !(swap*nop*nop, nop*swap*nop,
> >            !(swap*nop,nop*swap,filter1,nop*swap,swap*nop)
> >          * (swap, !(swap*nop,nop*swap,filter2))
> >        ),
> >   swap*nop)
> > 
> > Well, not very beautiful, but at least, it is possible to describe
> > such a filter chain.
> 
> interresting, lets add a 3rd one
> 
> (in0,tmp0,tmp1,tmp2)filter1(out0,tmp2,tmp8,tmp3);
> (tmp3,tmp4,in1,tmp5)filter2(tmp7,out1,tmp4,tmp0);
> (tmp6,in2,tmp7,tmp8)filter3(tmp5,tmp6,tmp1,out2)

There is no doubt that it's possible to convert it too.
And there is no doubt that it would be quite messy...

> And i wonder how many users will be able to do that ...

Not much, indeed. But I'm not sure many users would really
understand what this filter setup actually do, even with
your simple description. I personally have no idea what
can be achieved with such a filter setup.

> Also what about filters which can take a variable number of
> inputs/outputs?

Do such a beast exist ?
Anyway, I guess such a filter would simply accept as many input
the graph is feed it with.

> > I suppose this would not exactly be a common filter chain, and I
> > also suppose that filter1 and filter2 would naturally be designed
> > so that their "natural" self-feedback in and out pad would be the
> > first pad. This would highly simplify this filter graph. Here is an
> > example of the exact same graph, with ideally ordered filters pad:
> > 
> > (tmp0,tmp1,in0)filter1(tmp0,tmp2,out0);(tmp3,tmp2,in1)filter2(tmp3,tmp1,out1)
> > 
> > !(swap*nop, !( !filter1 * !filter2, nop*swap*nop, swap,nop,nop))
> > 
> > So IMO, this case just show that "mathematical" description allows
> > to represent complex graph as well as the original description.
> 
> no, it does not, it shows that you could represent the given graph.
> Its not a proof that all graphs can be represented. Though its quite
> easy to proof that if you want ...

Right.

> > But it don't show which one is the most simple/intuitive to
> > describe commonly used graph (ie. graph with no feedback or at
> > most one feedback).
> 
> Filter chains are just filter1,filter2,filter3 in either system.
> 
> 
> More complex ones without feedback could look like
> 
> filter1(T),filter2,filter3,(T)filter4
> VS.
> filter1,filter2*nop,filter3*nop,filter4
> 
> i think first wins

Maybe, but I don't see much difference.

Just a very slightly more complex example with no feedback:

filter1(T1),filter2,filter3,(T2)filter4;(T1)filter5(T2)
VS.
filter1,filter2*filter5,filter3*nop,filter4

To me, second syntax looks slightly more readable.

> With one feedback
> 
> (T)filter1,filter2,filter3(T),filter4
> VS.
> !(filter1,filter2,filter3),filter4

both looks equally simple.

> And with multiple feedbacks we have seen that the !*, description is
> significantly less readable

This I agree with.
But I wonder if multiple feedbacks do have enough practical usage
that we should care about their readability ?

> So IMHO while the * operator might be worth considering, the way
> feedbacks are handled is absolutely messy when there is more than 1
> feedback.

Agree.
BTW, both syntax are not mutually exclusive. Just replace () by []
in the original syntax and we could combine advantages.
Just an example with two "branches" plus a feedback:

(X)filter1(T1),filter2,filter3,(T2)filter4;(T1)filter5(X,T2)
VS.
[X]filter1,filter2*filter5[X],filter3*nop,filter4

> Also i think ,* is unintuitiv |, might be better but then maybe not i
> dunno

At first, * seemed not so bad to me, but | might be good too.

Aurel