[FFmpeg-devel] [RFC] Subtitle Filtering Ramp-Up

[softworkz .]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of Nicolas
> George
> Sent: Mittwoch, 4. Juni 2025 09:14
> To: FFmpeg development discussions and patches <ffmpeg-devel at ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [RFC] Subtitle Filtering Ramp-Up
> 
> Zach Swena (HE12025-06-03):
> > The way I see it, rudeness has been present on all sides of most of the
> > conflicts I have seen on the ML lately and it is way too easy to reflect
> > rudeness back so yes I think everyone on here needs to get over it and work
> > together politely.
> 
> How do you work politely with somebody who says “I am not insulting
> people calling them crazy because they are really crazy”?
> 
> > This is why I have been trying to ask high level questions.  His old
> > subtitle filtering patchset would need a lot of rework to bring to the
> > current codebase so starting over isn't a bad idea.  I don't really care
> > who works on or makes the commits for the code as long as the resulting
> > code is clean, makes sense to other developers and accomplishes what
> > everyone needs.  There are structural changes needed for what I want and it
> > would be good to find a solution that allows for functionality for
> > additional processing options also.
> 
> His old filtering patchet was broken beyond repair, and nothing changed.
> 
> Just to give an idea of my position on the topic: during one of the
> first VDDs, possibly the first one, I co-hosted with Ubitux a
> multi-hours brainstorming session on the matter of subtitles in
> libavfilter. That is how much I want to keep subtitles out of
> libavfilter, and that is how little time I have spent on it anticipating
> problems and finding solution.
> 
> When I say that softworkz's approach is broken, I know what I am
> talking about.
> 
> It is broken in three ways.
> 
> The first way is the hardest: it does not handle syncing, all the
> framesync code plus the complication of subtitles being sparse. It just
> feeds everything to libass and takes what comes out of it. It works when
> the subtitles arrive neatly before the video frames. It just does not
> work for such a simple use case as shifting the subtitles a few seconds
> forward.
> 
> softworkz has refused to even test that case.

I'd like to kind as you to provide such a case which my patchset 
doesn't handle. To make it easy, I could provide you a compiled 
binary for your convenience.


> But softworkz could not even test that case, because, second flaw, the
> easiest to fix: he neglected to implement all the utility filters, the
> filters that operate not on the frame payload but on timestamps, side
> data, other technical properties. All these filters are needed for
> subtitles too. softworkz patch series do not include them, and he
> refuses to implement them.

Which filters are you talking about specifically?

 
> It is a little harder than it looks, because implementing these filters
> by copy-pasting a third copy would not be acceptable, it must be done
> by factoring the existing duplicated code. But it being a little harder
> is not an excuse not to work at it.

I have no avoidance attitude towards hard work. Please let
me know which filters are you talking about.


> Third flaw: no negotiation. Negotiation is a core concept of
> libavfilter: have a RGB stream and a YUV-only filter? lavfi inserts the
> scale filter, and it inserts it at the right place. With softworkz's
> implementation: have text subtitles, expect bitmap ones, it fails
> instead of inserting the rasterizer at the right place.

Negotiation is supported and it is something different from auto-
insertion. For example, auto-insertion is happening to replicate 
the sub2video mechanism. Those command lines work just like before.

At one point I had said that I think it might be better when people
insert text2graphicsub filter manually, but auto-insertion is easily
doable, and I can add that with ease.



> This series only works in the simplest of test cases. It does not handle
> even one of the most obvious use cases, adjusting timing. It cannot even
> be called a proof of concept.

It has proven to work for a wide range of cases. Please provide a specific
example that should work but doesn't, then we can talk about it.


> The issue with this is that the negotiation code is extremely fragile
> and has barely any test coverage at all.

I had asked you many times about providing specific examples, but you 
never did. 

Please do that and we can talk about it.


Thanks
sw