[FFmpeg-devel] [PATCH] Add libavsequencer.

[Sebastian Vater]

Stefano Sabatini a ?crit :
> On date Wednesday 2010-08-18 22:54:06 +0200, Sebastian Vater encoded:
>   
>> Michael Niedermayer a ?crit :
>>     
> [...]
>   
>>> i think the problem we have is to get a single mixer and the surrounding
>>> code reviewed, understood and into svn currently.
>>> More mixers and a mixer selection api isnt going to help here.
>>> we are a bit short on reviewing man power for module related code
>>>   
>>>       
>> I know, that's why I thought (and agreed with Stefano) that we simply
>> start with the simplest version, the NULL mixer ;)
>>
>> The null mixer allows to review everything except the actual sample
>> reading stuff and optimized stuff. All (software) mixers will
>> practically work like this on a very basic manner. So reviewing this
>> will give basic understanding.
>>
>>     
>>> That said, there is nothing wrong with you working and improving what you
>>> like to. It doesnt help us to move forward with getting anything into svn
>>> though
>>>   
>>>       
>> It's just an idea I got while recently talking to Stefano regarding
>> this. So I thought to summarize our discussion regarding this here and
>> bring it to discussion.
>>
>> Still, it has nothing to do with the basic avseq integration patch, it
>> won't change regarding the final mixer API we will finally take
>> (regarding the hold back question).
>>
>> Maybe regarding the mixer stuff, could you give a suggestion, what I
>> should post here as [PATCH] for integration and what not?
>>     
>
> Trying to resume the main points here:
> 1) do we want a libavsequencer *independent* from libavcodec? Such a
>    library may be used for BSS definition / API and *eventually* for
>    mixer API. The latter should be moved to some other place in a
>    second moment, where it can be used indepentently from lavseq.
>   

Argumenting from the current size of the avsequencer size (compiled) it
makes sense (it's more than twice as large than libavfilter already).

So it's a good point for people don't needing it to disable it via
--disable-avsequencer configure option (hey mru, thank you very very
MUCH for your wonderful configure magic at this point!!!) and having all
that bunch out, this will also automatically disable all demuxers /
decoders requiring it.

>    Keeping BSS independant from lavc has also the advantage that for
>    example a MOD composer application won't depend on libavcodec, so I
>    tend to prefer this option.
>
> 2) BSS, which will go to libavsequencer, is still on a design
>    discussion phase. Maybe it's time to move the current discussion on
>    ffmpeg-soc here, where it would have more visibility.
>   

Is the ffmpeg-soc mailing list, as GSoC deadline has passed now, still
relevant?
I mean everything we do now, is beyond GSoC and therefore should be
moved to here, right?

> 3) As for the mixer API, there are many related problems. Do we want
>    such a complex API? I see the advantages of the design proposed by
>    Sebastian, but then I'm not really an audio expert so I don't feel
>    like I can't properly judge the implications of such a design.  The
>    location of the API is a secondary problem, since it can be moved
>    wherever we'll want in a second moment (libavmixer?,
>    libavresample?).
>   

Well, if you ask me, it is not really complicated as what the API
regards, as you see the structures definining it:
Currently AVSequencerMixerContext, the factory, with some few fields,
then we have AVSequencerMixerData the main structure returned by the
factory to work with which also has relative few fields and finally
AVSequencerMixerChannel which contains the data for each channel, which
is also pretty simple.

Especially if we compare that to the power they deliver.

I want to mention again the point that I want to have the mixer not only
to able to receive PCM data but from any decoder we have, and I wouldn't
wonder if it's enough looking at the current power of FFmpeg, it's
enough to replace the int16_t * field with an AVPacket * and that the
mixer simply calls get_buffer on it as the decoders do it?

The mixer works the way that it calculates the amount of samples to be
mixed in the next step for each channel in advance and then calls a
function with that amount, we could simply replace that with an
intermediate get_buffer call with this number of samples and then call
the actual PCM decoding as it does now.

Also don't get confused by the huge size of the current lq_mixer.c
implementation, this is not because it's soo complicated, it's reducable
to 2k lines of code with some #define magic, if we let mru do this, it's
even probably only 1k (seeing his neat tricks he was doing with IFF-ILBM
decoder lut).

>    What's for sure is that such an API will require a good amount of
>    time (indeed it could have been a GSoC task dedicated just for it),
>    if our objective is getting MOD support integrated as soon as
>    possible, then it would be better to rely on a much simpler API, so
>    this really depends on who will do most of the work (which is
>    likely Sebastian in this case), as for me I'll try to help as I can
>    in review (and mixing looks also required by audio lavfi).
>   

Well, I realized that, too. That way was too much for a single GSoC task ;)

> ...
>
> So in the end I suggest this course of action: apply the lavseq patch,
> *or* if we want to add more stuff to it before to commit it, then
> maybe we should focus on the BSS. Once that's ready, we can create
> lavseq *and* immediatly commit the BSS definition.
>   

One question, why that step (esp. the immediately after) should be
necessary?

> Alternatively we can work on the BSS *in libavcodec*, once it's ready
> we can create lavseq and move it to the new lib.
>
> The mixer path mainly depends on the contributors involved, on Michael
> and/or on who have ideas related to the advantages of such a design.
>
> Also I'd like to start to think about this eventual libavresample
> (required by audio/lavfi), we could eventually plan to add the mixer
> API there.
>