[FFmpeg-devel] [PATCH] libavcodec/nvdec: Do not exceed 32 surfaces when initializing hw_frames_ctx

[Ameer J]

From: ameerj <52414509+ameerj at users.noreply.github.com>

nvdec is likely to fail when the initial pool size exceeds 32. This change ensures we don't exceed the limit when initializing a new hw_frames_ctx

Signed-off-by: ameerj <52414509+ameerj at users.noreply.github.com>
---
 libavcodec/nvdec.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/libavcodec/nvdec.c b/libavcodec/nvdec.c
index 251be039a8..bef33dbae9 100644
--- a/libavcodec/nvdec.c
+++ b/libavcodec/nvdec.c
@@ -303,8 +303,10 @@ static int nvdec_init_hwframes(AVCodecContext *avctx, AVBufferRef **out_frames_r
     frames_ctx = (AVHWFramesContext*)(*out_frames_ref)->data;
 
     if (dummy) {
-        // Copied from ff_decode_get_hw_frames_ctx for compatibility
-        frames_ctx->initial_pool_size += 3;
+        // The function above guarantees 1 work surface, We must guarantee 4 work surfaces.
+        // (the absolute minimum), so add the missing count without exceeding the maximum
+        // recommended for nvdec.
+        frames_ctx->initial_pool_size = min(frames_ctx->initial_pool_size + 3, 32);
 
         frames_ctx->free = nvdec_free_dummy;
         frames_ctx->pool = av_buffer_pool_init(0, nvdec_alloc_dummy);
-- 
2.25.1