[MPlayer-dev-eng] Re: MCF ( Multimedia Container Format ) : Invitation to participate

[Christian HJ Wiesner]

Attention ! please use the 'reply all' function of your mail application to
make sure a copy is sent to the mailing lists of mplayer and MCF. If this
discussion is going on we might consider setting up a specific mailing list
titles 'mplayer-MCF-cooperaion' or similar.


My first mail to your mailing list :

< Dear mplayer development team,
< please allow me to introduce myself. My nick is ChristianHJW and
< some of you may perhaps know me from www.doom9.org where i am helping out
< as mod for the <'New A/V formats' section. I am also admin on
< www.powerdivx.com ( soon : www.corecodec.com ) and project admin for
< an opensource project called
< MCF ( Multimedia Container Format )
< http://mcf.sourceforge.net ; http://sf.net/projects/mcf
< MCF is ment to be an AVI replacement, but in contrary to the existing
< Ogg implementation from Tobias Waldvogel its not ment to be a windows
< based format ( <DirectShow implementation ) but was also targeting
< a x-platform compatibility from the very start.
< While the basic data and block structure is fixed now and already
< implemented in libmcf on CVS we are currently in the phase of deciding
< how to support all the different codecs and formats. Easiest thing to
< do what of course be to use the M$ dshow method of building a playback
< graph based on the FourCC of the <use codec. But as we are ware that
< this may cause problem on other OS such as Linux or MacOS we were
< discussing alternative solutions like a plugin <structure ( f.e. a
< MPEG4 ISO video plugin to play XviD and DivX5 ).
< Mplayer is currently the only Linux player that is capable of playing
< back .ogm files, so we are very interested how you managed to get this
< working. Is it possible to have dshow based parsers/decoders working on
< Linux ? Also we are convinced that the discussion between mplayer
< developers and our people could <lead to a very good cooperation for
< both sides. Good old AVI has definitely seen the end of its days, but
< we are not convinced that the existing Ogg <implementation is the answer
< to this. We hope MCF could raise your interest and look forward to
< your precious input.
< Please contact us on IRC, openprojects, #mcf to discuss how we could best
< implement codec/formats support into MCF.
< Best regards
< Christian
< P.S. Many thanks to KotH who was motivating us to start this discussion.


Your replies ( full threads here :
http://www.mplayerhq.hu/pipermail/mplayer-dev-eng/2002-June/008471.html ) :

Felix Buenemann :
> While the basic data and block structure is fixed now and already
> implemented in libmcf on CVS we are currently in the phase of deciding how
> to support all the different codecs and formats
I don't see where choosing the right video codec has anything to do with the
file format. The only thing the file format has to signal is, what codec is
used, the player can then use this info to choose the right decoder. Mostly
FOURCC would be enough for this, maybe an extra field that gives the long
name of the codec. And a thought would be to use 8 or 16 char codec
identifier, as four char seems to get close to it's limit with all those
different codecs out there. Exisitng codecs could use their FOURCC padded
with spaces to the new identifier size.
Also you should add a field to store codec specific data, this is eg. needed
to store the mpeg4 video headers.
All this was written out-of-the-blue, I haven't looked at the MCF specs yet.
> Mplayer is currently the only Linux player that is capable of playing back
> .ogm files, so we are very interested how you managed to get this working.
> Is it possible to have dshow based parsers/decoders working on Linux ?
We don't use the windows code, we have an own implementation of the ogm
demuxer. Btw. we can even use windows dshow codecs through our dshow
emulation layer
which is based on wine (code is inherited from aviplay project).
> Also we are convinced that the discussion between mplayer developers and
our
> people could lead to a very good cooperation for both sides. Good old AVI
> has definitely seen the end of its days, but we are not convinced that the
> existing Ogg implementation is the answer to this. We hope MCF could raise
> your interest and look forward to your precious input.
Yes, the ogg format wasn't at all aimed at video from the beginning, so it's
pretty much a hack/suboptimal from what I can see/have heard.
> Please contact us on IRC, openprojects, #mcf to discuss how we could best
> implement codec/formats support into MCF.
This would probably be a job for our demuxer-experts like AAA
(arpi,al3x,albeu
;-) and the ones I forgot. I did some demuxer hacking in mplayer, but I'm no
demuxer expert at all.
Btw. as arpi is lazy ;) it would probably be a better idea to visit us on
#MPlayerDEV in ircnet (irc.netsurf.de and others).
Or best thing keep the discussion on the mailinglist, this is the best way,
as
it allows all people to take part in the discussion and decisions. IRC is
more meant for quick-questions stuff.

My reply to Felix :

<I don't see where choosing the right video codec has anything to do with
the
<file format
Unfortunately its not that easy. What you say is certainly true for a format
that is mainly focussed on one particular OS, maybe two ( Windows, Linux ).
On Windows all you need is to provide the FourCC for a VfW or dshow based
player, and as most players on windows use a semi-automatic graph rendering
( again dhsow ) it wouldnt be too hard for us to simulate the behaviour of
AVI for this. In fact, the existing muxer for Ogg ( .ogm ) from Tobias is
based on Dshow also, Tobias created it to be used with Graphedit, being M$'
tool around Dshow. In principal it is demuxing an existing video stream from
an AVI, copying the codec related info from the AVI header ( WAVEFORMATEX ),
writing these into the Ogg header and thats it.
Its interesting to hear ( we truely didnt know ) that you can do the same
( or almost the same ) on Linux. But this still doesnt help for other OS
like MacOS, etc. and hardware support ( standalone players ). Especially the
last is a big problem, just look at the big varieties of different MPEG4V3
clones ( DivX3, SMR, Angelpotion, etc. ). All these can be played with the
same decoder, but all have different FourCC and GUIDs for codec
verification. In fact this is the main problem IMHO why there arent any
hardware players for DivX movies ... its the big variety of different
FourCCs and GUIDs being used for the very same thing. MCF is taking this
into account. We are planning to have
- a dshow mode (AVI compatibility mode ), using FourCC or GUID strings in
the format header to recognize the codec, plus having a AVI like rendering
function so every VfW or Dshow based player can use MCF once the format was
implemented into the player.
- a x-platform and hardware compatible 'plugin' mode, only for recognized
and standardized codecs ( like MPEG1,2,4, AAC, Vorbis, Tarkin, MP3, AC3,
etc. ). In this mode the decoding of the streams is not left to an existing
decoder or Dshow filter being installed on the PC, but is done with specific
plugins for each codec ( or better : compression format ), being created for
every OS. A first plugin could be a generic MPEG4 ISO video plugin to be
able to play DivX5, XviD, QT6, etc. A next plugin could be MP3, AC3, Vorbis,
whatever. Of course, this is big effort ! These OS specific plugins must be
specified, coded and maintained, being major work. But doing this we could
assure that an MCF file being created in plugin mode would run on every OS
with an OS specific, best suitable decoder, and even hardware support would
be much easier.
Its up to the user to decide on muxing the movie into MCF format whether he
would prefer Dshow or plugin mode. We have to tell him the pro's and con's
of each mode, like the ability to use the latest avaialble Dshow decoder for
his movie if he goes for Dshow mode, or x-platform and giid hardware
compatibility if he goes for plugin mode. On muxing we have to assume the
data provided are spec compatible, means our muxer app can not check if the
stream it gets is really MPEG4 ISO compliant.
< We don't use the windows code, we have an own implementation of the ogm
< demuxer. Btw. we can even use windows dshow codecs through our dshow
< emulation layer which is based on wine (code is inherited from aviplay
project).
Did Tobias help you with that, f.e. did he give you the basic structure of
his Dshow implementation, or could you do a kind of reverse engineering here
? I know he is using the basic data structure of Ogg, but we cant find
anything in the Ogg specs where the codec header info is stored. I had no
idea ( not sure about the MCF dev team, i am only a helping hand ) that
there is a possibility to emulate Dshow on Linux. Would you think of this as
a good solution ? Any know downsides ( high CPU power ) ?
< Yes, the ogg format wasn't at all aimed at video from the beginning,
< so it's pretty much a hack/suboptimal from what I can see/have heard.
Well, this is maybe not 100% true as Xiph people are working on Tarkin also,
being their own Video codec. But for sure it was not ment to hold any other
video data. Tobias Ogg implementation is in one way or another a kind of a
hack, yes. It was his decision how to put subtitles in Ogg, etc. . Also his
decision to copy the AVI header info and using FourCC for codec recognition
is maybe not a solution the Xiph people would like too much, as they are
aiming for x-platform compatibility also.


Arpi :

< my idea about the ideal file format:
< headers:
<  - should have headers. headerless formats (like mpeg) are very limited
<    (but may be edited using cut & dd :))
<  - should allow at least 256, but better if unlimited number of audio,
<    video and custom track types
<  - track header should have a 'type' field, telling the demuxer the format
<    of the header. why? currently, there are 3 big multimedia "platform":
<    windows (microsoft's vfw/dshow/activex) quicktime and realmedia.
<    they all have their set of codecs, and an own format (struct) of
<    audio/video/other track headers.
<    For windows, it's the BITMAPINFOSTRUCT (video) and WAVEFORMATEX
(audio).
<    For quicktime, it's ImageDescription (video) and SoundDescription(???)
(audio)
<    For real, there are also some struct, i don't know (didn't analized .rm
yet)
<    We could support these 3 at the beginning, while allowing other track
<    headers in the future. Why is it good? Both encoders and players
<    _have_to_ operate with these structs (codecs use it as in/output), so
<    it's easier to both player and encoder writers to use these, wihtout
<    converting them to some uniform struct and then trying to guess the
<    missing fields in the player. Disadvantage: a general 'print file info
<    only' tool have to know all these header structs. Or, we can add a very
<    simplified 'general' header (like AVI's AVIStreamHeader) containing
<    fourcc, width, height, color depth, aspect ratio, maybe bitrate,
<    quality, just for info.
< The actual way of implementing this can be the avi, asf or mov method,
< they all are ok (variable size chunks, starting with chunk size followed
by
< chunk type, so demuxers can skip unknown stuff, while allowing unlimited
< number of header defines)
< avi is the simplest, mov is over-complicated, and asf is the same as avi
but
< uses 16byte guids and 64bit for everything so just waste of disk space
< content:
< there is 3 kind of 'content formats' out there:
< 1. avi/mov/mpeg-pes style: variable size chunks, starting with chunk size.
<   raw seeking impossible, they usually have index table for seeking.
<   .mov doesn't have chunk header, it's unplayable without the index.
<   it isn't error-resistant at all, you can't re-sync if a chunk header is
<   broken, maybe try guessing and brute-force decoding but it's a mess
<2. asf, mpeg-ts (+ogg?) style: fixed size chunks, so raw seeking is
possible
<   (chunk position can be calculated). these chunks contain the real data
<   chunks fragmented. i like this approach, imho it's the best we can have:
<   raw seeking is possible without index, error-resistant, any size of the
<   real data chunk is allowed, no mess with sync-words.
<3. mpeg-ps/es style: fixed or variable sized chunks, separated by unique
<   sync-words (00 00 01 xx for mpeg). raw seeking is possible, but after
<   seeking, you have to search for the next sync-word, do some checks.
<   big disadvantage: the content of the embedded data chunks are limited -
<   they must _not_ contain the sync word! it's a real headache for codecs
<   not designed for this. advantage over 2.: resistant against problem when
<   parts of the file are shifted (few bytes missing) so the fixed-size
chunks
<   are all at wrong place so unrecoverable. may happen with streaming.
<   mpeg-ts fixes it by adding micro-sized header to each block, so such
<   shifting may be detected.
<a-v sync/timing:
<- fixed fps/sr - bad, forget it
<- timestamps at fixed clock (90khz for mpeg, 1ms for asf), limited, forget
it
<- timestamps at custom clock (used in .mov files) - recommended
<so, my opinion: we should NOT re-invent the wheel.
<there are great, well-designed (i bet lots of ppl worked a lot on these)
<file formats out there. they just don't apply our needs.
<they were designed for different purpose and different transfer media.
<but we could reuse the basid ideas of them, and combine the best part of
all
<file formats!
<it should have headers like avi, content like mpeg-ts and timing like in
mov


My reply to ARPI :

Thank you very much for your precious input. I will make sure our dev people
have a close look at yur suggestions, i am really not skilled enough to
comment on them.


Again ARPI :

<i've checked mcf after writting my mail, it's the same as TCF but renamed.
<it's no more than a little bit extended AVI, with variable size chunks and
<opendml-like hierarchical index :(
<
<so, i wote for the mplayer-fileformat then :)
<
< > so, i wote for the mplayer-fileformat then :)
> or change MCF like needed...
<
<i doubt it's possible...
<1, they already wrote some libs and apps using mcf,
<2, they announce mcf with teh most importamt feature: using minimum bits
for
<   headers, and maximize data area - it's against 2-layer packetization
<required for raw seeking and error-resistance. they don't address these
<issues, insteda they use index, and save index twice on theyr mode-2 cd
<format to "protect" it - it's silly imho, and won't work realiable
<one of their developers already forked, he also didn't accept this silly
<approach - maybe we should contact him. imho few MBytes for a 680MB movie
<does worth error-resistance and solid seeking... but their optinion is:
<don't waste bytes for useless headers...
<
<A'rpi / Astral & ESP-team

My reply to ARPI :

MCF was formerly called TMF ( The Movie Format ). This was dropped as
Pinnacle has a similar name for another thing.
About mplayer-fileformat : the reason to contact you was to make sure MCF
does meet your needs so you can support it.
Right now we can still change whatever we want, ther are no working MCF
muxers outside the dev team, no MCF files floating around. If you can give
us good reasons why our format is not as good as it should be you can rest
assured we will do whatever is necessary to make it the best format out
there.

Its not true we are willing to compromise the functionality and
compatibility of the format just to have a small header ! We just couldnt
find any reason until now why we should bloat header. If you can give us
good reason why to do it ( rememeber we want MCF to be the best format out
there ) we will certainly dont care about header size !

My contact with your people was done because right now we are in a stage
where we can still define a lot of things if necessary, so your input here
is highly appreciated.


Michael Niedermayer ( ein Bayer, wie ich ;-) ?? ):

>> i doubt it's possible...
>> 1, they already wrote some libs and apps using mcf,
>> 2, they announce mcf with teh most importamt feature: using minimum bits
>> for headers, and maximize data area - it's against 2-layer packetization
>> required for raw seeking and error-resistance. they don't address these
>> issues, insteda they use index, and save index twice on theyr mode-2 cd
>> format to "protect" it - it's silly imho, and won't work realiable one of
>> their developers already forked, he also didn't accept this silly
approach
>> - maybe we should contact him. imho few MBytes for a 680MB movie does
worth
>> error-resistance and solid seeking... but their optinion is: don't waste
>> bytes for useless headers...
>hmm, allthough i know very little about avi/asf/mcf/.../demuxers ...
>i dont really understand the problem with raw seeking vs small files as
>seeking is only meaningfull to keyframes, it should be enough to add a
fixed
>64 bit number per keyframe, this would increase the filesize only slightly
>(with a keyframe every 10sec its only +3kb per hour)
>and the probability that this 64bit value exists somewhere in the middle is
>so low that its not worth thinking about, allthough the completly paranoid
>might allso want to store the 64-bit value in tha main header instead of
>hardcoding it

My reply to Michael :

As i stated above, we are open for every fertile discussion.
Also you guys should take into account that our dev people have been
investing quite some time into how to create a new A/V format. Maybe there
are some very good reasons why to do it ... please wait for a mail of one of
our dev people.


Thanks to all of you

Christian