[MPlayer-dev-eng] NUT index proposal

[Michael Niedermayer]

Hi

On Wednesday 12 January 2005 19:11, D Richard Felker III wrote:
> On Wed, Jan 12, 2005 at 01:14:01PM +0100, Michael Niedermayer wrote:
> > Hi
> >
> > there are AFAIK 2 suggested index systems for NUT
> > 1. index which points to keyframes
> > 2. index which points to sync points and syncpoints which contain
> > pointers to previous keyframes
> >
> > 1. has a huge problem if the index is damaged and the keyframe distance
> > is large
> > 2. is quite complicated and has higher overhead
> >
> > i propose a 3rd variant, store multiple partial indexes pointing to
> > keyframes, each such index would be a list of (position, time, number of
> > keyframes since last entry) for each stream
> > each such index packet could be fixed size and updated later, though
> > neither of these is required, the index packets could very well just
> > contain previous keyframes
> > loosing a few such packets would not be a problem as only a few randomly
> > distributed keyframe pointers would be lost and they would be very likely
> > also be stored in another index packet
> >
> > another significant advantage is that seeking in incomplete/damaged files
> > is much faster, as without an index log2 n seeks are needed to find the
> > right position, if thats over a network 1-10sec per seek could be the
> > result, but with the partial index system a global time-position map is
> > available even in damaged files
>
> i'm not sure if i like this or not. for a non-corrupt, local file,
> seeking is at least slightly slower than with a unified index at the
> end, 

yes


> and for an app that wants to have the whole index available all 
> the time (e.g. a video editor) it would have to do lots of seeking
> throughout the file at load time.

yes, but the unified index, the way it was and is written in the spec would 
not contain all keyframes, so the situation after reading all or just a few 
of the partial indexes and reading the whole unified index would be somewhat 
similar 
the editor could also append a whole index at the fileend after reading all 
partial indexes once


>
> as for the other aspects of this, that might be good enough to
> outweigh the negative stuff; i'm just not sure. could you explain
> better what you mean by the above proposal, what these partial index
> packets would look like and how they'd fit into the nut bytestream,
> etc.?

well, i havnt thought about the details very much but i guess these packets 
should look like the index or info packets do, they also should be placed at 
predictable locations so they can be found easily


>
> one other concern i have, and this is entirely peripheral and nothing
> we've discussed before, but it did come up on irc once: it would be
> nice, given sequences of packets for each a/v/s stream, and given a
> framecode table and max_syncpoint_distance, if the nut file produced
> would be required to be unique. this would allow cool things like
> splitting and later reattaching segments of a file (e.g. to split
> across cds) without alterring the original. i don't know how you feel
> about this but i thought i'd mention it since lots of flexibility in
> how you generate the index sort of precludes this.

well iam not against this in principle but
* it would mean more difficult to implement and more complicated muxers
* some muxers will simply ignore these rules unless we require all demuxers to 
check for them and not play the file, then again this still could fail to 
enforce it if someone writes a muxer and demuxer together and that demuxer is 
used by enough people
* the point i fear most is that we will fail to consider all issues and the 
strict uniqueness would lead to some serious dissadvantage, just imagine 
quantization or motion estimation in mpeg to be similarly uniquely defined

[...]
-- 
Michael

"In any case, just because code is syntactically "valid" GNU C doesn't 
mean gcc can always compile it." -- justification to close a gcc bug