[FFmpeg-cvslog] avcodec/exr: Avoid signed overflow in displayWindow

[Michael Niedermayer]

ffmpeg | branch: master | Michael Niedermayer <michael at niedermayer.cc> | Mon Mar 21 21:03:13 2022 +0100| [1291568c9834c02413ab5d87762308f15b4ae9c6] | committer: Michael Niedermayer

avcodec/exr: Avoid signed overflow in displayWindow

The inputs are unused except for this computation so wraparound
does not give an attacker any extra values as they are already fully
controlled

Fixes: signed integer overflow: 0 - -2147483648 cannot be represented in type 'int'
Fixes: 45820/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_EXR_fuzzer-5766159019933696

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda at gmail.com>
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=1291568c9834c02413ab5d87762308f15b4ae9c6
---

 libavcodec/exr.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libavcodec/exr.c b/libavcodec/exr.c
index dd5924245f..f338ff0085 100644
--- a/libavcodec/exr.c
+++ b/libavcodec/exr.c
@@ -1833,8 +1833,8 @@ static int decode_header(EXRContext *s, AVFrame *frame)
             dx = bytestream2_get_le32(gb);
             dy = bytestream2_get_le32(gb);
 
-            s->w = dx - sx + 1;
-            s->h = dy - sy + 1;
+            s->w = (unsigned)dx - sx + 1;
+            s->h = (unsigned)dy - sy + 1;
 
             continue;
         } else if ((var_size = check_header_variable(s, "lineOrder",