[FFmpeg-devel] [PATCH][RFC] Lagarith Decoder.

[Vitor Sessak]

Nathan Caldwell wrote:
> 2009/9/5 Loren Merritt <lorenm at u.washington.edu>:
>> On Tue, 1 Sep 2009, Nathan Caldwell wrote:
>>
>>> Here are the latest lagarith patches.
>> Your decoder disagrees with the official version on the 2nd frame of http://akuvian.org/stuff/lagarith_bork.avi
> 
> Finally got a chance to look into this, and found the problem, but I'm
> not sure how to fix it.
> What happens is on this clip when the probabilities get scaled in
> lag_read_prob_header(), one of the values gets rounded up instead of
> rounding down like reference does. For the offending value reference
> gives 0x6ff while the code below gives 0x700.
> 
>         for (i = 1; i < 257; i++) {
>                 rac->prob[i] = ((uint64_t) rac->prob[i] <<
> scale_factor) / cumul_prob;
>             scaled_cumul_prob += rac->prob[i];
>         }
> 
> Subtracting 1 from the shifted probability forces it to round down,
> and fixes the issue on this clip, but breaks pretty much every other
> test clip I have. Here's what I did.
> 
>         for (i = 1; i < 257; i++) {
>             if (rac->prob[i] != 0)
>                 rac->prob[i] = (((uint64_t) rac->prob[i] <<
> scale_factor) - 1) / cumul_prob;
>             scaled_cumul_prob += rac->prob[i];
>         }
> 
> I'm hoping someone can help here, as I said I'm not sure how to fix this one.

I had a similar problem once and I solved it with a lookup table. See 
libavcodec/twinvq.c:very_broken_op().

-Vitor