[FFmpeg-cvslog] avcodec/osq: avoid signed overflow in downsample path

[Michael Niedermayer]

ffmpeg | branch: master | Michael Niedermayer <michael at niedermayer.cc> | Fri Jun 21 21:35:48 2024 +0200| [ed34b0c54ebdce7f741d9fb6a9ac11a1816df59c] | committer: Michael Niedermayer

avcodec/osq: avoid signed overflow in downsample path

Fixes: signed integer overflow: 865309950 * 256 cannot be represented in type 'int'
Fixes: 69191/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_OSQ_fuzzer-6310214413385728

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=ed34b0c54ebdce7f741d9fb6a9ac11a1816df59c
---

 libavcodec/osq.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/osq.c b/libavcodec/osq.c
index 7d0e8a87d5..1bd4485f07 100644
--- a/libavcodec/osq.c
+++ b/libavcodec/osq.c
@@ -321,7 +321,7 @@ static int do_decode(AVCodecContext *avctx, AVFrame *frame, int decorrelate, int
             cb->prev = prev;
 
             if (downsample)
-                dst[n] *= 256;
+                dst[n] *= 256U;
 
             dst[E] = dst[D];
             dst[D] = dst[C];