[FFmpeg-devel] [PATCH 2/5] avcodec/dstdec: Check for overflow in build_filter()

[Andreas Rheinhardt]

Rémi Denis-Courmont:
> Hi,
> 
> Down-casting to a signed type (here, int16_t) is implementation-defined. And while normal compilers do the expected thing, with modulo-2^n complement, sanitizers tend to dislike it.
> 

1. We expect the implementation-defined behaviour for signed types to
match the typical two's complement behaviour, see
https://ffmpeg.org/developer.html#C-language-features
2. In this case, there is no real-implementation-defined behaviour:
While the value (int16_t)v is implementation defined, whether it
coincides with v is not (it does so if and only if the value of v is
representable in an int16_t).
3. What sanitizers dislike it? After all, this is an explicit cast, so
e.g. UBSan would never report it as suspicious.

> AFAIK, the clean solution is via an union whence you assign the uint16_t member, and then read the int16_t member. Fortunately, GCC and LLVM are able to optimise that construct back to a single sign-extension.
> 

Type-punning via unions is implementation-defined, too. It will work
with two's complement types (as long as the number where all value bits
are unset and the sign bit is set is not a trap representation (all
exact-width intN_t are of this type)).

- Andreas