[FFmpeg-cvslog] swscale/output: Fix integer overflow in yuv2gbrp_full_X_c()

[Michael Niedermayer]

ffmpeg | branch: release/4.4 | Michael Niedermayer <michael at niedermayer.cc> | Tue Feb  4 03:58:44 2025 +0100| [e954f52713140e2d3963c1d08f25bed67fd5651e] | committer: Michael Niedermayer

swscale/output: Fix integer overflow in yuv2gbrp_full_X_c()

Fixes: signed integer overflow: 1966895953 + 210305024 cannot be represented in type 'int'
Fixes: 391921975/clusterfuzz-testcase-minimized-ffmpeg_SWS_fuzzer-5916798905548800

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit ce538ef97a7b1fdab6f2a3c8afc538c1cc3760d9)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=e954f52713140e2d3963c1d08f25bed67fd5651e
---

 libswscale/output.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libswscale/output.c b/libswscale/output.c
index e1c31a42e2..be22279229 100644
--- a/libswscale/output.c
+++ b/libswscale/output.c
@@ -2213,9 +2213,9 @@ yuv2gbrp_full_X_c(SwsContext *c, const int16_t *lumFilter,
         Y -= c->yuv2rgb_y_offset;
         Y *= c->yuv2rgb_y_coeff;
         Y += 1 << (SH-1);
-        R = Y + V * c->yuv2rgb_v2r_coeff;
-        G = Y + V * c->yuv2rgb_v2g_coeff + U * c->yuv2rgb_u2g_coeff;
-        B = Y +                            U * c->yuv2rgb_u2b_coeff;
+        R = Y + V * (unsigned)c->yuv2rgb_v2r_coeff;
+        G = Y + V * (unsigned)c->yuv2rgb_v2g_coeff + U * (unsigned)c->yuv2rgb_u2g_coeff;
+        B = Y +                                      U * (unsigned)c->yuv2rgb_u2b_coeff;
 
         if ((R | G | B) & 0xC0000000) {
             R = av_clip_uintp2(R, 30);