[FFmpeg-devel] [PATCH 06/11] avutil/half2float: adjust conversion of NaN

[Timo Rothenpieler]

On 10.08.2022 23:24, Andreas Rheinhardt wrote:
> Timo Rothenpieler:
>> IEEE-754 differentiates two different kind of NaNs.
>> Quiet and Signaling ones. They are differentiated by the MSB of the
>> mantissa.
>>
>> For whatever reason, actual hardware conversion of half to single always
>> sets the signaling bit to 1 if the mantissa is != 0, and to 0 if it's 0.
>> So our code has to follow suite or fate-testing hardware float16 will be
>> impossible.
> 
> What does the exr spec say about quiet and signaling nans?

Not sure how exr would be involved here.
But I tested this on both aarch64, x86 with sse2 emulation and x86 f16c 
on alderlake and zen2.
They all perfectly agree and match 100% what this changed code produces 
for the entire range of 65k possible values.