[FFmpeg-devel] SAMPLE_FMT_FLT bounds

[Vitor Sessak]

M?ns Rullg?rd wrote:
> Colin McQuillan <m.niloc at googlemail.com> writes:
> 
>> I'm looking for the correct way to clip the output of a
>> floating-point decoder.
>>
>> audioconvert.c converts floats thus:
>> lrintf(*(const float*)pi * (1<<15))) to convert to int16_t
>> lrintf(*(const float*)pi * (1<<7)) + 0x80) to convert to uint8_t
>>
>> qcelpdec.c clips to [-1.0, 32767.0/32768.0]. This is alright when
>> converting to 16-bit but will overflow when converting to 8-bit.
>> Should I clip to [-1, 127.0/128.0]? Or [-128.499/127, 127.499/128]?
> 
> IMO it would make more sense to combine clipping with conversion, as
> this can often be done in a single instruction.  If this means
> changing other decoders, so be it.

What I dislike about combining clipping and conversion is that IMO 
decoders that output floats should output sane values. If some last 
filtering step is sometimes unstable and outputs +inf, it should be 
clipped in the decoder. So, for such a decoder, if clipping and 
conversion were merged, one would end up clipping twice (one for 
sanitize and other for whatever format we are converting to).

A second problem is if a format do not need clipping (all values never 
are outside the "good" range), clipping would be useless and inefficient.

>> The attached patch is just an idea.
>>
>> Index: libavcodec/qcelpdec.c
>> ===================================================================
>> --- libavcodec/qcelpdec.c	(revision 19594)
>> +++ libavcodec/qcelpdec.c	(working copy)
>> @@ -826,8 +826,8 @@
>>  
>>      formant_mem = q->formant_mem + 10;
>>      for(i=0; i<160; i++)
>> -        *outbuffer++ = av_clipf(*formant_mem++, QCELP_CLIP_LOWER_BOUND,
>> -                                QCELP_CLIP_UPPER_BOUND);
>> +        *outbuffer++ = av_clipf(*formant_mem++, SAMPLE_FMT_FLT_MIN,
>> +                                                SAMPLE_FMT_FLT_MAX);
> 
> This should be simdified.

This is indeed somewhat speed-critical. Does the implementation you have 
in mind uses cmov?

-Vitor