[FFmpeg-devel] [PATCH] libavutil/softfloat: Fix normalizations.

[Michael Niedermayer]

On Thu, Jun 04, 2015 at 12:45:52PM +0000, Nedeljko Babic wrote:
> >> diff --git a/libavutil/softfloat.h b/libavutil/softfloat.h
> >> index 8097d28..182e517 100644
> >> --- a/libavutil/softfloat.h
> >> +++ b/libavutil/softfloat.h
> >> @@ -47,7 +47,7 @@ static const SoftFloat FLOAT_0999999    = { 0x3FFFFBCE,   0};
> >>  static av_const SoftFloat av_normalize_sf(SoftFloat a){
> >>      if(a.mant){
> >>  #if 1
> >> -        while((a.mant + 0x20000000U)<0x40000000U){
> >> +        while((FFABS(a.mant) + 0x20000000U)<0x40000000U){
> >
> >when exactly is this needed and how does a.mant reach that value
> >for which this is needed?
> >FFABS would significantly slow this down i suspect and it just looks
> >wrong
> >
> 
> The problem occurs for some negative numbers during normalization.
> 
> For example, please consider if 0xFFFFFFD80 is sent to av_normalize_sf.

considering that this is not a 32bit value and mant is, theres no
way that can happen


> 
> The while loop has a condition: (a.mant + 0x20000000U)<0x40000000U.
> 0xFFFFD80 is cast to unsigned and we have:

and that lost its highest 8bit
also FFABS() does not chaneg the result for this


[...]
> >
> >>              a.mant += a.mant;
> >>              a.exp  -= 1;
> >>          }
> >> @@ -68,7 +68,7 @@ static av_const SoftFloat av_normalize_sf(SoftFloat a){
> >>  
> >>  static inline av_const SoftFloat av_normalize1_sf(SoftFloat a){
> >>  #if 1
> >> -    if((int32_t)(a.mant + 0x40000000U) < 0){
> >> +    if((int32_t)(a.mant + 0x40000000U) <= 0){
> >
> >    0x40000000 + 0x40000000U == 0x80000000 not 0 so this shouldnt
> >make a difference
> 
> There is a typo in comment.
> It should be : "when mantis is -0x4000000".

applied this part

[...]

thanks
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Republics decline into democracies and democracies degenerate into
despotisms. -- Aristotle
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20150604/a0bda15f/attachment.asc>