[FFmpeg-devel] [RFC] libavfilter audio API and related issues

[Måns Rullgård]

Stefano Sabatini <stefano.sabatini-lala at poste.it> writes:

>> 3. The current output sample format is always forced to S16. The
>> channel layout conversion filters after this all assume that the
>> individual samples are shorts. This needs to be fixed right? For
>> example, I was thinking of changing stereo_to_mono function so it
>> accepts an extra stride parameter and uses that to do its job
>> irrespective of whether inputs are shorts (S16) or otherwise. Is
>> this ok?
>
> Yes and yes, and I would like to hear someone which worked extensively
> with the audio API to say which are the potential problems/warts to
> keep in mind for such an implementation.
>
> Resuming:
> * audioconvert.h/audioconvert.c:
>     struct AVAudioConvert
>     generic sample and channel layout API
>     av_audio_convert* API, supports up to 6 channels

This interface is mostly good.  The implementation, however, is
abysmally slow.  It is on my much-too-long todo list to fix this.
Help would be appreciated.

> * avcodec.h/resample.c
>     struct ReSampleContext
>     audio_resample_init() - deprecated
>     av_audio_resample_init()
>     audio_resample()
>     audio_resample_close()

I thought this was meant to be deprecated and replaced with the one below.

> * avcodec.h/resample2.c
>     struct AVResampleContext
>     av_resample_init()
>     av_resample_close()
>     av_resample_compensate()
>     av_resample()

This a high-quality resampler which even has a (compile-time)
"audiophile kiddy mode".

If keeping the old one for speed reasons is desired, the fast
resamplers should be moved behind the same interface as the
high-quality ones.  This would of course require adding a parameter to
av_resample_init() for choosing the algorithm, much like swscale has.

> Can someone say which is the main difference between the various APIs
> and what's the plan for their support (i.e. which is going to be
> deprecated in the future?).
>
> This API seems to need anyway some (major?) rework, I already proposed
> some time ago to create a separate lib for this (libavresample -
> lavre?), which would have a role similar to that of libswscale.

That might make sense.  It's not a decision we have to make
immediately though.

>> I still need to fix nits etc in af_resample. Will do those when
>> finishing the channel layout conversion.
>
> For the moment I believe that we need to focus on getting the first
> simple version of the audio framework in place, then we will hopefully
> work through the audio resampling framework current limitations.

Fully agree.  I would leave resampling alone for now.  There is plenty
of work to be done on a generic audio framework, from defining structs
to represent audio packets through to generic audio support in
libavfilter.

-- 
M?ns Rullg?rd
mans at mansr.com