[FFmpeg-devel] [PATCH 4/5] avcodec/fitsdec: Use lrint()

[James Almer]

On 12/5/2019 2:46 PM, Michael Niedermayer wrote:
> On Mon, Sep 30, 2019 at 10:30:59PM -0300, James Almer wrote:
>> On 9/30/2019 1:30 PM, Michael Niedermayer wrote:
>>> Fixes: -nan is outside the range of representable values of type 'unsigned short'
>>
>> From lrint documentation:
>>
>> "If x is a NaN or an infinity, or the rounded value is too large to be
>> stored in a long (long long in the case of the ll* functions), then a
>> domain error occurs, and the return value is unspecified."
>>
>> So i don't know if using lrint is a good idea here.
>>
>>> Fixes: 17769/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FITS_fuzzer-5678314672357376
>>
>> Is the output of av_int2double/av_int2float or header.data_min NaN in
>> that testcase? Wouldn't it be better to check that instead, and abort?
> 
> This patch also fixes the fate failures on 32bit x86 fitsdec-bitpix-64
> 
> I can add per pixel checks but i expect that not to fix fate
> so i suggest to apply this as a fate fix first and then we can decide if we
> need an additional fix for NaN
> 
> thx

Fine by me.