[MPlayer-dev-eng] Flaming NUT

[D Richard Felker III]

On Tue, May 04, 2004 at 01:25:19AM +0300, Ivan Kalvachev wrote:
> 
> Hello,
> Maybe you have noticed that I am not including myself in nut discussions anymore.
> Why?
> I looked at older (20030906) version on nut and I found that I like it
> more, as it is nearly what I have proposed. From this fact I think that
> nut is heading in a wrong direction. Michael is great optimizer,
> but I will repeat again (and again) that reducing overhead is not our
> primary goal!

This is not about overhead, altho the old design SUCKED and had
considerably worse overhead than most other containers, unless you
used subpackets. But subpackets preclude live streaming, and live
low-bitrate streaming is where overhead matters most!!

If you notice, all the design changes we have made recently in NUT
fulfill MULTIPLE goals, which include:

- allowing zero-latency muxing (streaming)
- reducing overhead
- improving error resilience

> I found something interesting too. The old draft have more
> requirements for error resistance! Cheaters.)

Yes, it had one more requirement: the ability to identify damaged
frame _contents_ in order to re-download. We decided this is stupid
because it belongs at the protocol level (ed2k, bittorrent, rtsp, tcp,
whatever).

> So, I will try to explain one more time my point of view.
> 
> 1. The format is not extendible.
> "allow adding more fields at the end of headers".
> This also include frame headers!

Not to us. The frame headers are designed not to need more fields. We
will not make further additions (which would prevent old players from
supporting new NUT files) after the spec is final and in-use!

> After all, the main function of container is to keep the frame data
> and all other meta data that is required for it.
> 
> Just turn back and see what beautiful hack you have done for DTS support.

Actually the dts "hack" is only there so we can formally specify what
proper interleaving means in the spec. In practice, a demuxer/decoder
never needs to know dts. Instead it just uses "delay" as the decoding
latency, and thus demuxes "delay" packets ahead at all times, which
will give the same effect.

So, our "beautiful hack" is really just a clever way of (formally)
writing down what everyone has known for years.

> Oh I am sure that you will may extend frame_code to add additional bits
> that will indicate additional fields. Maybe frame_codes won't be
> enough?
> I guess that escape_frame_code (e.g. 0xFF) could be used, so you will
> still have about 250 normal codes. Of course escaped code will grow to
> minimum 2 bytes  :( Not to mention main and stream header changes...

Maybe you have some points about the current structure not being
extensible enough. I'll check and recommend any necessary changes. IMO
there should be a number before the framecode table, which tells how
many vlc-coded fields will follow for each framecode. That way we can
add new fields that can be ignored by old players.

> 2. I wanted more checksums (in frame headers too). Rich explained me
> that they are not necessary as backward/forward pointers are the
> biggest part of the header and breaking one of them is easy to be
> found. Few days later Michael removed the backward pointers.
> Of course nobody want to spend  precious bytes for checksums.

Checksums ARE NOT NECESSARY to identify damage. Startcode checking
does a much better job of the same thing, with less computational
overhead (I sure as hell don't want to be wasting cpu cycles on
checksums when I'm trying to get my k6 to decode 720x480 video with
he-aac audio.....).

> 3. Seeking ATM is something horrible. The place of index is not
> determined by any way. It may come to the confusing situation to read
> the half (or whole) file until index is found:O

There is no good spec for the index right now, so your complaint is
about the incompleteness of NUT, not a flaw in NUT. Rest assured we
WILL specify the index properly, but not necessarily before the "1.0"
NUT spec. Seeking in NUT is very efficient without an index.

> Seeking without index is very similar to the mpeg stream seeking.

Nope. Correct seeking in mpeg is O(n) because timestamps are entirely
random. In practice mpeg players implement incorrect bitrate-based
seeking, which usually seeks by the wrong amount (thus everyone
complaining about -ss with mpeg files...).

Seeking in NUT with no index is O(log n). You always seek to the exact
time you want.

> With the tiny detail that nothing guarantee that these startcodes are
> unique, just very improbable.

Guess what? Nothing guarantees that a "packet" with the right checksum
is valid either. It's just incredibly improbable to get a bad packet
with a good checksum. The probability of a false positive for your 32
bit checksum is 1/2^32. The probability of false positive for NUT
startcodes is 1/2^64.

> Do you remember the nut it nut scenario.

Nut is not allowed to contain other containers (including itself),
only raw codec frames.

> It could make nut demuxer
> completely nuts on seeking, dumping errors that don't really exist.
> Ya,ya even ogg don't do it anymore.

You can come up with pathologies that break any system of sync/error
recovery.

> 4. I am serious about NUT in MPEG-TS. Really.
> You are trying to reduce the overhead of NUT, with words that it is
> good for very low bitrate streams. But these streams are good only for
> streaming. And NUT is not suited for streaming. No packet priority
> no retransmition etc...

This belongs in the PROTOCOL, not the file format. They are two
entirely separate layers.

> 5. The current level of error resistance ability is somehow lower
> than MPEG-(P)ES. Why? If there is broken frame_header of type0 frame
> then we will loose all frames until next type2. This may mean up to
> 0.5 seconds (15 frames) or even more (>16kb) data.

Nonsense. See below for explanations.

> Just for compartment with mpeg elementary stream you may resync on next
> slice in the same picture!

You can do this just fine with nut. If the damage is only inside the
frame (and not in the nut headers), then the codec does its own
resyncing with error concealment, and you'll never notice any problem.

Also, if you know your nut file contains (for example) only mp3 audio
and mpeg-4 video, you can use mpeg headers to resync. BUT THIS IS
CODEC-SPECIFIC, and so is the resyncing you're talking about in
MPEG-ES. It does not work in general.

> If some player need error recovery it is better to completely ignore NUT
> demuxer and search mpeg startcodes in the data.

You're free to do this for MPEG files if you like...

> 6. There is no way to find error in the frame headers or frame data,
> unless something really bad happens (hit impossible value). Even then
> the damaged region could not be exactly determined :(

Yes it can. That's what recovery points (aka type1 startcodes) are
for. If the chain of packet lengths does not lead to a recovery point
within the coded max_distance, then the header is invalid. Otherwise
it's assumed valid (probability of it being invalid is very very low).

Recovery points are sufficiently small (only 3 bytes!) that you can
have the interval between them be very small (e.g. 4k), and then it's
easy for the demuxer to check them in its buffers. If the interval is
too large for the demuxer, it can instead remember the last known
"good" point, and then go back and check (via seeking) if it goes too
long without finding any startcodes. OR, a basic demuxer can simply
wait until the next sync point to begin demuxing again, which is NOT A
BIG LOSS!!

> Do you remember the scenario I was trying to convince you while I was
> studding NUT? It was not possible because of backwards pointers. But now
> it is:
> If there is one broken byte in frame_code of type0, it is very probable
> that demuxer won't detect it and will return data_frames. It may
> calculate wrong size of frame and will read wrong frame_code for next
> frame. Same will repeat until summary size grows beyond the limit. But
> as the size is variable now, it could become quite big mess.

Look Ivan. Error resilience is UP TO THE PERSON CREATING THE FILE. If
you want to avoid losing anything when there's an error, you're
perfectly free to put a startcode before each and every frame. Nothing
is stopping you. And this may be useful for archival material.

On the other hand, if you're copying a DVD to share on P2P ;), or
streaming video (where the PROTOCOL ALREADY HAS ERROR DETECTION AND
RETRANSMISSION), then you and use very few startcodes, since you have
other ways of repairing the data if it's damaged.

> 7. The size test are done in best conditions. We don't have real test
> stream that could be used for comparing results. Every test is unique
> and unrepeatable. Of course nut will beat every container in CBR mp3, except
> mp3, as it have monotone timestamp
> s, (nearly) constant
> framesize and only one stream. So what?
> We need an real stress test, for worst scenario.

Stress test is 8kbit/sec vorbis. I demonstrated that the base overhead
for this case is 1 byte-per-packet. You can't get lower than that. Add
whatever level of error resilience you like on top, but IMO 8kbit/sec
is normally for streaming where the PROTOCOL ALREADY RECOVERS ERRORS!

> Non of these problems are fatal. Most of the time nut will work.

It will always work for what it's intended for, by the person making
the file. Forcing people who have no use for extensive error
resilience to waste tons of kabbebytes on it is idiotic. What's even
worse is forcing people to waste tons of kabbebytes on USELESS
overhead that doesn't even help with error recovery (MKV and OGG).

> Making container better than avi is not big deal.

Tell that to the OGG people... :)) They still can't seem to do it.

> Don't forget that OGG is quite good container and only one

ROTFL!!!

> small (design) error banned it from broad usage. Now there is OGG2 in
> development and if there is even single problem in NUT it may be doomed.
> (Not to mention matroska;)

FUD.

> You know that a chain is as strong as it weakest branch.

The word is link.

> Best Regards
>    Ivan Kalvachev
>   iive
> 
> p.s.
> Yes, it is possible to create container with even lower overhead, that
> is more simple and error resistant. But it looks like all current nut's
> are against any kind of buffering.(

Feel free to explain how. IMO it can't be done even with buffering.

> But I want to delay that project after I hove some working vo3.
> 
> p.s.2
> sorry I missed matroska flaming...

:)

Rich