[Ffmpeg-devel] Re: BUG: AC3 encode volume is low

[Justin Ruggles]

Michael Niedermayer wrote:
> On Sat, Apr 08, 2006 at 08:59:26AM -0400, Justin Ruggles wrote:
>>Why does the MDCT output to a 32-bit int instead of 16-bit?  Isn't it
>>just a signed 15-bit fixed-point implementation of the formula in the
>>spec?  I can't find where the output coefficients would need to be more
>>than a 16-bit signed integer.
> 
> 
> and thats what scares me, i dont know ac3 very well, but i do know that a
> N point decorrelation style transform will generally need log2(sqrt(N)) 
> bits more at the output to maintain the same precission
> 
> 
> 
>>>From what I can tell from the MDCT code, and from the fix15() function,
>>the output coefficients should have a range of -32767 to 32767.  Thus,
> 
> 
> i dont think this is true

I've looked into this a little more now.

Using floating-point math in the 512-point MDCT reference equation, and
taking the window function into account, it seems that the maximum range
of the coefficients is +/- ~0.752.  This would give a fixed-point range
of +/- 24641.

I've attached the test program I used to come to this conclusion.  If
there is something I'm missing just let me know.

-Justin