[MPlayer-dev-eng] NUT infodata proposal [updated]

[D Richard Felker III]

actually i lied in the subject, this isn't a full proposal, but some
ideas to start with. basically i'm going to try to classify some types
of info that people might (hypothetically, somewhere down the road)
want to store in a multimedia file, and what the requirements are for
storing that info in a sane way:



catalogue info:
title, author, artist, stars, publisher, description, genre,
releasegroup, year, duration, ...

this type of information is very desirable to most users, but
otherwise nonessential, in the sense that it's perfectly possible to
enjoy the file without it. in general this data is not specific to one
stream, but applies to the file as a whole (or at least to a range
within the file).


capture info:
creationtime, source, capturedevice, ...

rather unimportant. this could very well be considered per-stream
data, but if all the streams were captured together one might want to
put the data in a combined packet.


stream relationship info:
language, disposition, ...?

information describing the relationships between the streams in a
file. actually many complicated relationships are possible (e.g.
"stream 3 is subtitles for the parts of stream 0 that are spoken in
german") but trying to represent complicated relationships might just
overcomplicate things and discourage people from storing any data. at
the very least you need to know whether text and audio streams are
dialogue or "extra" in nature. you might also need to know for video
streams, e.g. if you have a music file with both live concert footage
and a music video as video streams. without this information or user
intervention, it will be very difficult for a player to sanely pick
which streams to present.


navigation info:
chapters, ...?

information used to assist the user in seeking or choosing a part of
the multimedia file to play. useless unless it is easily accessible to
the player at load time, so that the player can present the user with
choices of where to seek. i am strongly against including any sort of
editlist or menu functionality in here, since it will only serve to
cripple players that want to play linear movies, and since it's
abusive of the "one file, one unit of content" principle. we already
have a mess with quicktime and edit data inside the movie files (not
to mention other nasty data like redirections). i do not want to see
that kind of thing in nut.


content description info:
???

identification of people/animas/objects in video, descriptions of
scenes, assistance for the blind or visually impaired to interpret the
movie, ... there are many possibilities. the problem is this sort of
data is very difficult to store, and the optimal storage method might
depend on what the intended purpose is. for instance if you have an
educational film about animals and want to find the scenes in it that
contain a particular animal, you might wish to have information like
this at the beginning of the file. but if you just want to be able to
have names appear next to the picture (or when you click on the
animal) as the movie is playing, you might want the info interleaved
with the movie, with per-frame polygon outlines like michael
suggested.



ok, with all that in mind, i see:

some data that needs to be immediately available to be useful at all
(stream relationship, navigation info). we have to find a way to put
this at locations where it can be immediately found by the player or
we might as well not have it.

some data needs to be spread throughout the streams (catalogue info in
streaming content, content description info).

some data that would be more useful if available immediately, but
which is generally nonessential (catalogue info, capture info, perhaps
some types of content description info).



keep in mind that i'm not necessarily proposing we use the info
packets already written in the spec for this stuff. for interleaved
stuff, we might actually want to use a dedicated stream so it can have
timestamps...a stream where each timestamped packet contains an info
packet, or something like that. i'm not sure. but i think this whole
issue is a lot more complicated than any of us are giving it credit
for (myself definitely included!) and we'd better think it out much
better before finalizing the spec.


rich