[FFmpeg-devel] [PATCH 2/2] avutil/executor: Fix stack overflow due to recursive call

[Zhao Zhili]

> On Jul 8, 2024, at 22:07, Timo Rothenpieler <timo at rothenpieler.org> wrote:
> 
> On 08.07.2024 09:43, Zhao Zhili wrote:
>> From: Zhao Zhili <zhilizhao at tencent.com>
>> av_executor_execute run the task directly when thread is disabled.
>> The task can schedule a new task by call av_executor_execute. This
>> forms an implicit recursive call. This patch removed the recursive
>> call.
>> ---
>>  libavutil/executor.c | 5 +++++
>>  1 file changed, 5 insertions(+)
>> diff --git a/libavutil/executor.c b/libavutil/executor.c
>> index 89058fab2f..c145c51be9 100644
>> --- a/libavutil/executor.c
>> +++ b/libavutil/executor.c
>> @@ -58,6 +58,7 @@ struct AVExecutor {
>>      int die;
>>        AVTask *tasks;
>> +    int stack_depth;
>>  };
>>    static AVTask* remove_task(AVTask **prev, AVTask *t)
>> @@ -207,8 +208,12 @@ void av_executor_execute(AVExecutor *e, AVTask *t)
>>      }
>>        if (!e->thread_count || !HAVE_THREADS) {
>> +        if (e->stack_depth > 0)
>> +            return;
>> +        e->stack_depth++;
>>          // We are running in a single-threaded environment, so we must handle all tasks ourselves
>>          while (run_one_task(e, e->local_contexts))
>>              /* nothing */;
>> +        e->stack_depth--;
> 
> Won't this put the above line into the "nothing" while-loop?
> Is that intended? If so, the indentation should be adjusted accordingly.
> If not, the while-loop should gain empty {}.

The comment specify it’s a while loop with empty body. Maybe it’s not obvious
in the email client.

> 
>>      }
>>  }
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".