[FFmpeg-devel] [RFC] fftools/ffmpeg and libavdevice/sdl issue

[Anton Khirnov]

Quoting Zhao Zhili (2023-12-13 11:37:52)
> 
> 
> > On Dec 13, 2023, at 18:06, Anton Khirnov <anton at khirnov.net> wrote:
> > 
> > Quoting Zhao Zhili (2023-12-13 10:31:38)
> >> 
> >>> On Dec 13, 2023, at 17:08, Anton Khirnov <anton at khirnov.net> wrote:
> >>> 
> >>> Quoting Zhao Zhili (2023-12-12 18:27:39)
> >>>> Now it's time to talk about the libavdevice/sdl issue.
> >>>> 
> >>>> SDL output is broken with ffmpeg multithread refactor. SDL 'muxer' write_header
> >>>> and write_packet must be run in the same thread. And to make it work portable
> >>>> and reliable, SDL 'muxer' must be run in main thread. It's a common requirement
> >>>> for render to be run in main thread.
> >>>> 
> >>>> There are at least two trac tickets for the same issue: #10644 and #10649.
> >>>> 
> >>>> And there are two patches for the issue:
> >>>> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20230918063728.198377-1-haihao.xiang@intel.com/
> >>>> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20231101090115.10655-1-angus.chen@intel.com/
> >>>> 
> >>>> There patches might work on Linux, but not portable.
> >>>> 
> >>>> A simple workaround is pipe the output of ffmpeg to ffplay:
> >>>> 
> >>>> ./ffmpeg -re -i /e/video/cctv.mp4 -an -f yuv4mpegpipe - |ffplay -
> >>>> 
> >>>> To fix it, another thread can be used to drive transcode rather than main thread.
> >>>> A main loop should be created on main thread, and prepared to handle any special
> >>>> tasks like render. It sounds a lot of work. I'm not sure if it worth for a single use case.
> >>>> However, maybe we can have a libavfilter/vsink_preview after that.
> >>>> 
> >>>> What should we do?
> >>> 
> >>> Honestly I don't see how this could be done in ffmpeg CLI without
> >>> disgusting hacks, but before that the question is: why is there an SDL
> >>> "muxer" and why would anyone want to use it in ffmpeg CLI? What actual
> >>> use cases does it serve that cannot be better handled otherwise?
> >> 
> >> I pasted wrong tickets, they are #10625 and #10649.
> >> https://trac.ffmpeg.org/ticket/10625
> >> 
> >> The use case is realtime preview. The function lavd/sdl2 provides is limited. A vsink_preview
> >> filter is more appropriate. It has the same thread issue with libavfilter.
> > 
> > The submitter claims that piping to ffplay suffers from latency, which
> > should not be there and so is either a bug or an improper setup.
> 
> The latency issue may be real or not, I’m concerned with performance. Colorspace is easy to
> be handled in the same process than pipe to ffplay, although it’s missing in current implementation.

I would think piping yuv4mpeg to ffplay should have very little overhead
compared to actual encoding.

IMO ffmpeg is just not a GUI program and trying to use it as one will
just unavoidably lead to tears. It's not an email client either btw.

-- 
Anton Khirnov