[FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Zhao Zhili]

> On Dec 13, 2023, at 18:06, Anton Khirnov <anton at khirnov.net> wrote:
> 
> Quoting Zhao Zhili (2023-12-13 10:31:38)
>> 
>>> On Dec 13, 2023, at 17:08, Anton Khirnov <anton at khirnov.net> wrote:
>>> 
>>> Quoting Zhao Zhili (2023-12-12 18:27:39)
>>>> Now it's time to talk about the libavdevice/sdl issue.
>>>> 
>>>> SDL output is broken with ffmpeg multithread refactor. SDL 'muxer' write_header
>>>> and write_packet must be run in the same thread. And to make it work portable
>>>> and reliable, SDL 'muxer' must be run in main thread. It's a common requirement
>>>> for render to be run in main thread.
>>>> 
>>>> There are at least two trac tickets for the same issue: #10644 and #10649.
>>>> 
>>>> And there are two patches for the issue:
>>>> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230918063728.198377-1-haihao.xiang@intel.com/
>>>> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20231101090115.10655-1-angus.chen@intel.com/
>>>> 
>>>> There patches might work on Linux, but not portable.
>>>> 
>>>> A simple workaround is pipe the output of ffmpeg to ffplay:
>>>> 
>>>> ./ffmpeg -re -i /e/video/cctv.mp4 -an -f yuv4mpegpipe - |ffplay -
>>>> 
>>>> To fix it, another thread can be used to drive transcode rather than main thread.
>>>> A main loop should be created on main thread, and prepared to handle any special
>>>> tasks like render. It sounds a lot of work. I'm not sure if it worth for a single use case.
>>>> However, maybe we can have a libavfilter/vsink_preview after that.
>>>> 
>>>> What should we do?
>>> 
>>> Honestly I don't see how this could be done in ffmpeg CLI without
>>> disgusting hacks, but before that the question is: why is there an SDL
>>> "muxer" and why would anyone want to use it in ffmpeg CLI? What actual
>>> use cases does it serve that cannot be better handled otherwise?
>> 
>> I pasted wrong tickets, they are #10625 and #10649.
>> https://trac.ffmpeg.org/ticket/10625
>> 
>> The use case is realtime preview. The function lavd/sdl2 provides is limited. A vsink_preview
>> filter is more appropriate. It has the same thread issue with libavfilter.
> 
> The submitter claims that piping to ffplay suffers from latency, which
> should not be there and so is either a bug or an improper setup.

The latency issue may be real or not, I’m concerned with performance. Colorspace is easy to
be handled in the same process than pipe to ffplay, although it’s missing in current implementation.

> 
> -- 
> Anton Khirnov
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org <mailto:ffmpeg-devel at ffmpeg.org>
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org <mailto:ffmpeg-devel-request at ffmpeg.org> with subject "unsubscribe".