[FFmpeg-cvslog] avcodec/gemdec: Check tag & planes before image allocation

[Michael Niedermayer]

ffmpeg | branch: master | Michael Niedermayer <michael at niedermayer.cc> | Sat Nov 27 21:00:07 2021 +0100| [6e34d947978867789ca563aaced26b166753b062] | committer: Michael Niedermayer

avcodec/gemdec: Check tag & planes before image allocation

Fixes: Timeout
Fixes: 41083/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_GEM_fuzzer-5843826518917120

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Peter Ross <pross at xvid.org>
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=6e34d947978867789ca563aaced26b166753b062
---

 libavcodec/gemdec.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/libavcodec/gemdec.c b/libavcodec/gemdec.c
index 2464dda8a7..eee21a50d4 100644
--- a/libavcodec/gemdec.c
+++ b/libavcodec/gemdec.c
@@ -157,6 +157,14 @@ static int gem_decode_frame(AVCodecContext *avctx,
     if (header_size >= 11)
         tag = bytestream2_peek_be32(&gb);
 
+    if (tag == AV_RB32("STTT") || tag == AV_RB32("TIMG") || tag == AV_RB32("XIMG") ||
+        planes == 1 || planes == 2 || planes == 3 || planes == 4 ||
+        planes == 8 || planes == 16 || planes == 24) {
+    } else {
+        avpriv_request_sample(avctx, "planes=%d", planes);
+        return AVERROR_PATCHWELCOME;
+    }
+
     if ((ret = ff_get_buffer(avctx, p, 0)) < 0)
         return ret;
 
@@ -236,9 +244,6 @@ static int gem_decode_frame(AVCodecContext *avctx,
         planes = 1;
         row_width = avctx->width * pixel_size;
         put_lines = put_lines_bytes;
-    } else {
-        avpriv_request_sample(avctx, "planes=%d", planes);
-        return AVERROR_PATCHWELCOME;
     }
 
     ret = av_reallocp_array(&avctx->priv_data, planes, row_width);