[FFmpeg-devel] Integrating the mod engine into FFmpeg - what is the best design approach?

[Sebastian Vater]

Michael Niedermayer a ?crit :
> On Thu, Aug 05, 2010 at 02:54:07AM +0200, Stefano Sabatini wrote:
>   
>> OK that's a nice description of the whole BSS design.
>>
>> Sebastian is currently working on this git branch:
>> http://github.com/BastyCDGS/ffmpeg-soc.git
>>
>>     
>>> Open discussion points are:
>>> 1. Best way of integration into rest of FFmpeg
>>>       
>> I'm resuming some of the designs which has been already proposed:
>> please correct me if some information is missing / uncorrect.
>>
>> 1)
>>  The MOD decoder does just one thing: decode a AVPacket to a BSS. It
>>  does not know anything about the player (it doesn't even know _if_ it
>>  will be played or converted to other format or fed to a visualization code).
>>  3- Libavsequencer does just one thing: transforming a BSS in PCM audio.
>>  It knows nothing about file formats (it don't care or know if the BSS
>>  was made from a MOD file or recorded from a MIDI keyboard).
>>
>>  That's why we insist in starting with the implementation of MOD ->  XM
>>  conversion: it is much simpler than MOD ->  PCM conversion, it doesn't
>>  need an implementation of libavsequencer.
>>
>>                            mod file - metadata                      BSS +
>>                                                               sequencer SAMPLES
>>  MOD file -->  MOD demuxer -------------------->  MOD decoder  ------------------>  application
>>
>>  Advantages of this approach as follows:
>>  - Allows for conversion from a format with more features to one with
>>  less doing no mixing or sampling
>>  - Makes each file format very modular (just reading the bitstream and
>>  filling up BSS)
>>  - Better integration with the way FFmpeg works ATM
>>     
>
> A Audio decoder should return PCM. Doing something else is requireing all
> applications that use libav to be changed. I dont see the point in going this
> way
>   

This is what I realized up to now, but I realized also that this
connection doesn't only start of the decoding part but also on the
demuxing part (see lavf/iff.c vs. lavc/8svx.c) which I decided to take
as a base.

I just commited some code which partially has in lavf the IFF-TCM1
detection and forwarding it to lavc/seq_tcm.c.

This causes more trouble, because I just have to figure out how to set
ByteIOStream for these, so I can evaluate the IFF chunks as in lavf.

> There are the whole avsequencer apis that allow full and direct access to
> all things. The existing audeio decoder API seems sufficient for what it is
> intended for, namely decoding audio. We arent returning SAMPLE_FMT_MDCT either
> for aac.
>
> If you want to implement convertion between xm->mod before decoding to pcm
> thats ok but the BSS goes to a field of AVCodecContext or AVFrame and PCM
> samples could be set to 0
> thats in line of how video transcoding with reusing motion vectors is done
>   

I currently want for simplicity having the BSS/player output always
SAMPLE_FMT_S32 (that's the output of the mixers anyway) for raw audio stuff.

>
>   
>> 2)
>>  The demuxer decodes the file to a BSS an output it in an
>>  AVPacket. It would them define a CODEC_ID_SEQUENCER, and the decoder
>>  would be just a wrapper to libavsequencer to make the BSS -> PCM
>>  conversion.
>>
>>  The advantage of this approach is that the concept of demuxing/decoder
>>  does not make much sense for these formats, so this avoid the
>>  artificial distriction. Moreover, it makes a nice distinction of
>>  transcoding from one MOD format to other (with -acodec copy) to
>>  decoding it to PCM. The disadvantages is that API-wise it's less clear
>>  for external applications to get the BSS data (reading the AVPacket
>>  payload). Besides, all the bit-reading API is part of lavc.
>>     
>
> This is unacceptable
>   

I just made some thoughts about AVMEDIA_TYPE_SEQUENCER instead of
AVMEDIA_TYPE_AUDIO if someone requests conversation, is that better?

-- 

Best regards,
                   :-) Basty/CDGS (-: