[FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Zhao Zhili]

Now it's time to talk about the libavdevice/sdl issue.

SDL output is broken with ffmpeg multithread refactor. SDL 'muxer' write_header
and write_packet must be run in the same thread. And to make it work portable
and reliable, SDL 'muxer' must be run in main thread. It's a common requirement
for render to be run in main thread.

There are at least two trac tickets for the same issue: #10644 and #10649.

And there are two patches for the issue:
https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230918063728.198377-1-haihao.xiang@intel.com/
https://patchwork.ffmpeg.org/project/ffmpeg/patch/20231101090115.10655-1-angus.chen@intel.com/

There patches might work on Linux, but not portable.

A simple workaround is pipe the output of ffmpeg to ffplay:

./ffmpeg -re -i /e/video/cctv.mp4 -an -f yuv4mpegpipe - |ffplay -

To fix it, another thread can be used to drive transcode rather than main thread.
A main loop should be created on main thread, and prepared to handle any special
tasks like render. It sounds a lot of work. I'm not sure if it worth for a single use case.
However, maybe we can have a libavfilter/vsink_preview after that.

What should we do?