[NUT-devel] Info packets in NUT stream (spec bugs?)

[Michael Niedermayer]

Hi

On Wed, Nov 15, 2006 at 11:29:12PM +0100, Michael Niedermayer wrote:
> Hi
> 
> On Wed, Nov 15, 2006 at 07:44:15AM +0200, Oded Shimon wrote:
> > On Wed, Nov 15, 2006 at 03:07:49AM +0100, Michael Niedermayer wrote:
> > > On Wed, Nov 15, 2006 at 02:17:03AM +0200, Oded Shimon wrote:
> > > > On Tue, Nov 14, 2006 at 07:04:43PM +0200, Oded Shimon wrote:
> > > > > On Sun, Nov 12, 2006 at 05:27:06PM +0200, Oded Shimon wrote:
> > > > > > On Fri, Nov 10, 2006 at 08:57:23AM +0200, Oded Shimon wrote:
> > > > > > > This is a new proposal: Basically identical to the previous one, only info 
> > > > > > > packets SHOULD NOT appear "in the wild" in non-realtime-streams instead of 
> > > > > > > MUST NOT. Making the distinction between file and realtime streams less 
> > > > > > > strict. Demuxers still SHOULD NOT search for info packets anywhere except 
> > > > > > > after the main headers. I think this is most clean...
> > > > > > > 
> > > > > > > Comments?
> > > > > > 
> > > > > > Repost. ... 48 hour notice ...
> > > > > 
> > > > > Committed
> > > > 
> > > > OK, I'm sorry I committed this so hastily. You said you object until you 
> > > > have heard from Rich. I talked to Rich on IRC and he seemed apathetic on 
> > > > the issue and unwilling to reply on it. I changed my proposal to be 
> > > > somewhat saner with a less strict distinction between realtime and 
> > > > non-realtime streams, and you have not made any objection reply since.
> > > > 
> > > > Can we discuss this now?
> > > 
> > > yes we can, what about keeping the current info after header + info must
> > > be identical rule
> > > with an exception like
> > > if a nut "file" is transmitted over the network and no out of band method
> > > to update metadata is available then info packets may be placed between
> > > ?syncpoints? the contents of these info packets may change and they do
> > > override the info packets after the mainheader. for the normal info
> > > packets after the mainheaders the normal rules apply (=no changes)
> > > dumping such a network stream to disk does not constitute a valid nut
> > > file, to convert such a stream to a valid nut file it is needed to
> > > read the whole to find the most up to date info and then place this
> > > after every main/stream header group
> > > 
> > > to simplify this conversation every of the "new" info packets must
> > > contain a pointer to the previous "new" info packet?
> > 
> > Do you suggest adding some sort of flag to info packets saying if they are 
> > "new" ones or "header" ones?
> 
> no their position already defines that clearly
> 
> 
> > 
> > > and some approproate stuffing/empty info packet should be added
> > > after all main/stream headers?
> > 
> > Well, for one, this stuffing packet would violate the "identical info 
> > packets after main header" rule. 
> 
> no it would not, my proposal would have putted the pointers only in the
> new changing packets
> 
> 
> > Also, the main headers are in the 
> > begginning, and a smart network stream would have no reason to repeat 
> > them, so they will not point backwards to anything...
> 
> if they are not repeated then its not a valid nut file
> 
> 
> > 
> > I don't think the pointer stuff helps at all - either the stream dumper is 
> > NUT aware, in which case it can rip out all the info packets found during 
> > dumping (and optionally add them to main headers somehow), or it is not 
> > NUT aware and it doesn't care anyway.
> > 
> > You seem to be scared of people putting info packets in the middle of 
> > files or in dumped realtime streams - I don't see this as an issue. 
> > Demuxers SHOULD NOT search for info packets anywhere except after the main 
> > header. If you were silly enough to stick an info packet somewhere in the 
> > middle, a demuxer will not see it until playbacking that point (you 
> > SHOULD NOT do this). And in the case of dumped realtime streams, you can 
> > fix them fairly easily by remuxing...
> 
> you can fix nut files on writeable media, but if its allowed to have
> such packets in nut files (compared to just streams sent over the network)
> then theres a good chance that dvds with nut files (assuming they would
> ever be used on dvds) would missuse that ...
> 
> really IMO streaming and filestoreage are different and applying the
> same rules to both will cause disadvantages in both, slightly seperate 
> rules make more sense
> 
> repeating headers in a stream is unneeded if you can gurantee error free
> transmission (either by retransmits like TCP or just a silly dissconnect-
> reconnect) for a file which can be stored and transmitted over unpredictable
> channels the repeats make sense, they also make sense in non specific
> streaming ...
> changing info makes sense for streaming but in files stored on your dvd
> its a PITA
> you could now try to invent some messy system (info streams ...) to make
> changed info findable in files on your dvd but thats a ugly hack to force
> one solution to cover 2 cases (for the network case theres no need to
> find these packets ...)
> 
> 
> really i think that nut-np is what should be used, if you want to use nut
> over tcp, thats fine but then live with its limitations and dont just
> hack it because you cant live with 1 or 2 pages of code to handle a minimal
> additional header from nut-np over TCP
> 
> also you dont need syncpoints over TCP, its a waste of bandwidth ...

heres a simple nut streaming spec proposal (whats not explicitly overriden
is as defined by the current nut spec)

streaming in general:
* nut is transmitted raw (no extra headers)
* info may change and be transmitted anywhere
* the client can (if supported by the server do seeks, ask for header or
  info retransmitts, and ask for intra refresh (keyframe), and request
  a syncpoint)

streaming over error free channels:
* main, stream and info headers MUST NOT be repeateded unless requested by
  the client and supported by the server
* there is only 1 syncpoint after the headers unless server side seeking
  is supported and used
-----
streaming over channels with packet loss:
* packet boundaries are aligned with nut frame starts where possible
* packet boundaries are aligned with slice starts where possible
* FLAG_CODED_PTS SHOULD be set for all frames which use a frame from
  a previous packet to calculate the pts
* if the packet starts with a framecode then it MUST have FLAG_CHECKSUM
  set (that way packets which continue past packets can be distinguished
  from packets starting new frames)


[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

In the past you could go to a library and read, borrow or copy any book
Today you'd get arrested for mere telling someone where the library is