[MPlayer-dev-eng] NUT error resilience ideas

[D Richard Felker III]

On Mon, May 10, 2004 at 02:23:58AM +0200, Michael Niedermayer wrote:
> Hi
> 
> idea 1:
> store sum of size of 2 packets (current+last) instead of only the current size 
> and store the size of the last packet at a syncpoint
> 
> advantages: 
> * we could walk backward from a syncpoint, allthough its not exactly obvious 
> how far we should walk backwards, as its difficult to distinguish damaged 
> from undamaged packets
> 
> disadvantages: 
> * brute force resync without a syncpoint would be very difficult
> * larger packet headers, especially if small and large packets are mixed
> 
> overall, i doubt this would be a good idea, but i thought i mention it ...

Agree, this one sucks.

> idea 2:
> duplicate all packet headers in an optional index at the end of the file, so 
> the index is just packet headers but no packets, and some alternative value 
> for the startcodes, but is otherwise identical to the actual stuff before it
> 
> advantage:
> * if the specific part of the index is undamaged, we can easily resync to the 
> next undamaged packet
> 
> disadvantage:
> * if the index is used it adds some extra size at the end, while a keyframe or 
> synccode only index would be significantly smaller
> 
> IMHO idea 2 seems to be a good idea

I'd rather make it a separate "recovery file". That way users who
don't want/need it can ignore it without having to modify the main nut
file (which is bad for p2p use ;).

> idea 3.
> this one has low overhead but high complexity, probably too high ...
> we first split the file in blocks of n packets (n may be different for each 
> block) we could for simplicity also assume to split exactly at syncpoints.
> each block will get an error recovery record added at its end, otherwise its 
> identical to the current spec, so encoding and decoding have low latency.
> to build the error recovery record we first build an array of all the packet 
> headers within the block, then we calculate some parity data (RS or 
> whatever), this parity data is then stored at the end of each block
> 
> so how can we use this now during decoding?
> its easy in theory ...
> we simply demux until we notice an error

Ivan believes it's hard to notice an error. What do you think?

> and then do a bruteforce resync to 
> find the most likely packet chain up to the next syncpoint, now we have a 
> damaged list of packet headers and the error recovery record from prior of 
> the syncpoint, next is pretty obvious, use the error recovery record and any 
> additional information about which packet headers are lost to repair them, 
> after that just demux the not yet demuxed parts of the block

Hmm, IMO it will be hard to get satisfactory results from the recovery
codes if brute forcing didn't pick the right chain to begin with. And
if it did pick the right chain, we probably don't need recovery codes
anyway...

Rich