[FFmpeg-cvslog] avcodec/vc2enc: basic sanity check on slice_max_bytes

[Michael Niedermayer]

ffmpeg | branch: release/4.2 | Michael Niedermayer <michael at niedermayer.cc> | Thu Sep 12 20:03:55 2024 +0200| [0b22eccc7cf7df8ac2a04874f2cf275161cec0ee] | committer: Michael Niedermayer

avcodec/vc2enc: basic sanity check on slice_max_bytes

Fixes: left shift of 896021632 by 3 places cannot be represented in type 'int'
Fixes: 70544/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VC2_fuzzer-6685593652756480

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit 6df9a0292ca6c29ef3b220fbf9b257924cabf035)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=0b22eccc7cf7df8ac2a04874f2cf275161cec0ee
---

 libavcodec/vc2enc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/vc2enc.c b/libavcodec/vc2enc.c
index 774cce80db..a3057f91fb 100644
--- a/libavcodec/vc2enc.c
+++ b/libavcodec/vc2enc.c
@@ -984,7 +984,7 @@ static av_cold int vc2_encode_frame(AVCodecContext *avctx, AVPacket *avpkt,
     }
 
     s->slice_min_bytes = s->slice_max_bytes - s->slice_max_bytes*(s->tolerance/100.0f);
-    if (s->slice_min_bytes < 0)
+    if (s->slice_min_bytes < 0 || s->slice_max_bytes > INT_MAX >> 3)
         return AVERROR(EINVAL);
 
     ret = encode_frame(s, avpkt, frame, aux_data, header_size, s->interlaced);