[FFmpeg-devel] [PATCH 2/2] avutil/executor: Fix stack overflow due to recursive call

[Timo Rothenpieler]

On 08.07.2024 09:43, Zhao Zhili wrote:
> From: Zhao Zhili <zhilizhao at tencent.com>
> 
> av_executor_execute run the task directly when thread is disabled.
> The task can schedule a new task by call av_executor_execute. This
> forms an implicit recursive call. This patch removed the recursive
> call.
> ---
>   libavutil/executor.c | 5 +++++
>   1 file changed, 5 insertions(+)
> 
> diff --git a/libavutil/executor.c b/libavutil/executor.c
> index 89058fab2f..c145c51be9 100644
> --- a/libavutil/executor.c
> +++ b/libavutil/executor.c
> @@ -58,6 +58,7 @@ struct AVExecutor {
>       int die;
>   
>       AVTask *tasks;
> +    int stack_depth;
>   };
>   
>   static AVTask* remove_task(AVTask **prev, AVTask *t)
> @@ -207,8 +208,12 @@ void av_executor_execute(AVExecutor *e, AVTask *t)
>       }
>   
>       if (!e->thread_count || !HAVE_THREADS) {
> +        if (e->stack_depth > 0)
> +            return;
> +        e->stack_depth++;
>           // We are running in a single-threaded environment, so we must handle all tasks ourselves
>           while (run_one_task(e, e->local_contexts))
>               /* nothing */;
> +        e->stack_depth--;

Won't this put the above line into the "nothing" while-loop?
Is that intended? If so, the indentation should be adjusted accordingly.
If not, the while-loop should gain empty {}.

>       }
>   }