[FFmpeg-devel] [PATCH] ac3enc: fix 16-bit signed overflow in mdct
Justin Ruggles
justin.ruggles
Tue Dec 21 22:12:38 CET 2010
On 12/21/2010 10:51 AM, Justin Ruggles wrote:
> On 12/20/2010 07:47 PM, Michael Niedermayer wrote:
>
>> On Mon, Dec 20, 2010 at 11:40:51PM +0000, Mans Rullgard wrote:
>>
>> Welcome back! :)
>>
>>
>>> One of the cos table values is -32768 and must not be negated.
>>> ---
>>> libavcodec/ac3enc.c | 2 +-
>>> 1 files changed, 1 insertions(+), 1 deletions(-)
>>>
>>> diff --git a/libavcodec/ac3enc.c b/libavcodec/ac3enc.c
>>> index d372e6a..1191d16 100644
>>> --- a/libavcodec/ac3enc.c
>>> +++ b/libavcodec/ac3enc.c
>>> @@ -405,7 +405,7 @@ static void mdct512(AC3MDCTContext *mdct, int32_t *out, int16_t *in)
>>> for (i = 0; i < n4; i++) {
>>> re = ((int)rot[ 2*i] - (int)rot[ n-1-2*i]) >> 1;
>>> im = -((int)rot[n2+2*i] - (int)rot[n2-1-2*i]) >> 1;
>>> - CMUL(x[i].re, x[i].im, re, im, -mdct->xcos1[i], mdct->xsin1[i]);
>>> + CMUL(x[i].re, x[i].im, -re, -im, mdct->xcos1[i], -mdct->xsin1[i]);
>>
>> this contains more negates than before and i suspect the amount can be reduced
>> if it is slower
>
>
> I slightly prefer to change the table generation back to clipping -32767
> to 32767. Anyone against?
I'm going to go ahead and change this back. There is no quality
difference. Any speed difference doesn't matter because it is only used
in initialization.
-Justin
More information about the ffmpeg-devel
mailing list