[FFmpeg-devel] [PATCH] avcodec/decode: clean-up if get_hw_frames_parameters fails

[Thomas Guillem]

On Mon, Oct 7, 2024, at 17:12, Rémi Denis-Courmont wrote:
> Le 30 septembre 2024 17:27:46 GMT+09:00, Thomas Guillem via 
> ffmpeg-devel <ffmpeg-devel at ffmpeg.org> a écrit :
>>Fixes the following assert:
>>
>>[00007f1df83d17e0] vaapi generic error: avcodec_get_hw_frames_parameters failed: -22
>>Assertion p_dst->hwaccel_threadsafe || (!dst->hwaccel && !dst->internal->hwaccel_priv_data) failed at libavcodec/pthread_frame.c:349
>>
>>Reproduced from VLC with VAAPI, when fallbacking from hw to sw.
>>---
>> libavcodec/decode.c | 7 +++++++
>> 1 file changed, 7 insertions(+)
>>
>>diff --git a/libavcodec/decode.c b/libavcodec/decode.c
>>index c331bb8596..e90a6630e1 100644
>>--- a/libavcodec/decode.c
>>+++ b/libavcodec/decode.c
>>@@ -19,6 +19,7 @@
>>  */
>> 
>> #include <stdint.h>
>>+#include <stdbool.h>
>> #include <string.h>
>> 
>> #include "config.h"
>>@@ -1173,6 +1174,7 @@ int avcodec_get_hw_frames_parameters(AVCodecContext *avctx,
>>     const AVCodecHWConfigInternal *hw_config;
>>     const FFHWAccel *hwa;
>>     int i, ret;
>>+    bool clean_priv_data = false;
>> 
>>     for (i = 0;; i++) {
>>         hw_config = ffcodec(avctx->codec)->hw_configs[i];
>>@@ -1197,6 +1199,7 @@ int avcodec_get_hw_frames_parameters(AVCodecContext *avctx,
>>             av_buffer_unref(&frames_ref);
>>             return AVERROR(ENOMEM);
>>         }
>>+        clean_priv_data = true;
>>     }
>> 
>>     ret = hwa->frame_params(avctx, frames_ref);
>>@@ -1217,6 +1220,10 @@ int avcodec_get_hw_frames_parameters(AVCodecContext *avctx,
>> 
>>         *out_frames_ref = frames_ref;
>>     } else {
>>+        if (clean_priv_data) {
>>+            av_freep(avctx->internal->hwaccel_priv_data);
>>+            avctx->internal->hwaccel_priv_data = NULL;
>
> Isn't this assignment redundant? Though IMO, `av_freep` is an 
> abomination that needs to be exterminated with extreme prejudice, but I 
> digress. Did you mean to use `av_free`?

Ah yes, I will fix it. I will use `av_freep` in my future patch since this is what is currently used for this variable in the current file.

>
>>+        }
>>         av_buffer_unref(&frames_ref);
>>     }
>>     return ret;
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".