[FFmpeg-devel] [PATCH] seperate parser from codec selection

[Paul Kendall]

On Tuesday 26 May 2009 11:34:05 pm Michael Niedermayer wrote:
> On Tue, May 26, 2009 at 12:23:55PM +0200, Michael Niedermayer wrote:
> > On Tue, May 26, 2009 at 06:56:14PM +1200, Paul Kendall wrote:
> > > On Tuesday 26 May 2009 02:02:09 pm Michael Niedermayer wrote:
> > > > On Sun, May 17, 2009 at 03:25:22PM +1200, Paul Kendall wrote:
> > > > > This patch allows an AVStream to specify a parser_id separately to
> > > > > the codec_id. This will be useful for streams such as DVB-T which
> > > > > has AAC audio which may be contained in an LATM multiplexed stream.
> > > > >
> > > > > The patch will fall-back to using the codec_id for parser selection
> > > > > if the parser_id is set to CODEC_ID_NONE, which is the default.
> > > >
> > > > I do not like having 2 codec ids.
> > > >
> > > > [...]
> > > >
> > > > >      enum CodecID codec_id; /* see CODEC_ID_xxx */
> > > > > +    enum CodecID parser_id; /* see CODEC_ID_xxx */
> > > >
> > > > even less so without any documentation
> > > >
> > > > [...]
> > >
> > > Micheal, would it be preferable to make a new enum for parsers such as
> > > PARSER_ID_xxx?
> >
> > no
> >
> > fields are not just random numbers to be hacked as needed for each issue
> > without thought about the semantical effect.
> >
> > codec_id represents the format of the bitstream
> >
> > Either it is the same for the various AAC variants, in which case all
> > code claiming to support CODEC_ID_AAC should also support all AAC
> > variants (as they are implicated by the codec_id)
> >
> > OR
> >
> > there are 2 AAC codec_ids in which case there are at least 2 options
> > A. find a way to chain demuers cleanly, each would have its own codec_id
> >    clearly seperated
> > B. extend the API of bitstream filters so they can change codec_id and
> >    add have a pre and post automatic bitstream filtering codec_id, this
> >    requires review and adjustment of all code using codec_id
> >
> > There maybe other options, but CODEC_ID_AAC cannot mean different things
> > depending on where its used that would be a too obscure design, that
> > noone would understand.
>
> in the long run, being able to chain demuxers would surely be the nicest
> and most powerfull, and it might against my expectation not really harder
> than the other solutions as we keep running into issues with them ...
>
> To chain demuxer we would need the following:
> 1. a generic combining demuxer, that being just a simple demuxer that
>    passes its read_packet/read_header to its parent demuxer(s)
>    The purpose of this one is just to turn the AVFormatContexts and
> AVStreams of more than 1 demuxer into a single AVFormatContext with its
> AVStreams so the user can use several demuxers through the single demuxer
> API. 2. We need a LATM demuxer (which should be very trivial) its input
> just being LATM and its output just being a single [or in the future >1] no
> LATM AAC AVStream
> 3. We need a URLProtocol or ByteIOContext that takes the packets a demuxer
>    outputs as its input.
> 4. The whole chain could then be setup by the core simply by checking if
>    codec_id of any stream is one that needs a demuxer.
>
>  _________             ________                                        
> _________
>
> |         | -stream0->|        |-----------------------------stream0-->|   
> |         |      |-stream0->
> |         |
> |         |           |        |   _____________    _______            |   
> |         |           |        |      |
> |         |           |        |
> |         |           |        |  |Packet to    |  |LATM   |-stream1a->|   
> |         |           |        |  |      |-stream1->
> |
> | MPEG-PS | -stream1->| Packet |->|ByteIOContext|->|Demuxer|          
> | |Combining| Demuxer |           | Buffer |  |_____________| 
> | |_______|-stream1b->| Demuxer |-stream2->
> |
> |_________|
> | -stream2->|________|-----------------------------stream2-->|_________|-st
> |ream3->
>
> Comments, especially from members of other project that do support chaining
> demuxers is welcome!
>
>
> [...]
Micheal, this looks to be a very sensible approach to the problem.

As an example of how this would work to get things rolling.

An mpeg-ts file with a stream of LATM wrapped AAC data.
The mpeg-ts demuxer creates 3 streams, h.264, LATM data, and subtitles.
h.264 stream has codec_id H264, the latm/AAC stream would have AAC as the
codec_id and also set that a LATM demuxer needs to be used. Is this correct?

If so, then the AVStream structure needs a demuxer_id field or some such.

Next, because it has a demuxer_id in the stream, and demuxers take a
ByteIOContext as input, a Packet to ByteIOContext converter is used to stream
the data to the LATM demuxer, which returns packets. All of the packets from
the various AVFormatContexts and hence AVStreams are then passed to the
combining demuxer which creates a single AVFormatContext as output with
all the streams from the input AVFormatContexts.

So in this method, there is no bit stream filter for LATM. But you should still
be able to insert them at any point along the chain.

Please let me know if I have this right, as I can and would like to start to
work on this.

Cheers,
Paul