[FFmpeg-devel] [PATCH 1/7] avcodec/pnmenc: Check av_image_get_buffer_size()

[Andreas Rheinhardt]

Fixes the crash in ticket #10050.
Also ensure that we don't overflow before ff_get_encode_buffer().

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt at outlook.com>
---
 libavcodec/pnmenc.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/libavcodec/pnmenc.c b/libavcodec/pnmenc.c
index 9eb663306d..c998dd410c 100644
--- a/libavcodec/pnmenc.c
+++ b/libavcodec/pnmenc.c
@@ -42,7 +42,10 @@ static int pnm_encode_frame(AVCodecContext *avctx, AVPacket *pkt,
     int size = av_image_get_buffer_size(avctx->pix_fmt,
                                         avctx->width, avctx->height, 1);
 
-    if ((ret = ff_get_encode_buffer(avctx, pkt, size + 200, 0)) < 0)
+    if (size < 0)
+        return size;
+
+    if ((ret = ff_get_encode_buffer(avctx, pkt, size + 200U, 0)) < 0)
         return ret;
 
     bytestream_start =
-- 
2.34.1