[FFmpeg-devel] Status and Plans for Subtitle Filters

[Clément Bœsch]

On Thu, Feb 27, 2020 at 12:35:03PM +0100, Anton Khirnov wrote:
[...]
> AFAIU one of the still-open questions for the subtitle redesign is what
> does it mean to decode or encode a subtitle.

There are multiple markups available for text subtitles, and there are
multiple ways of representing graphic rectangles for bitmap subtitles.

So for text subtitles, decoding and encoding respectively means
transforming them in a common markup to represent them all (currently ASS
is our "raw" representation) and back into their markup specifications. We
have a bunch of those already (subrip, microdvd, subviewer, ...).

For bitmap subtitles, decoding and encoding respectively means
transforming the bitstream into rectangle structures with RAW images
inside and back into the codec-specific bitstream.

> And one of the options is putting the AVPacket->"decoded subtitle"
> (whatever that is) and "decoded subtitle"->AVPacket conversions into a
> separate library.

And then you can't have them in libavfilter, so you can't have a sane
harmony with medias including subtitle streams. It's problematic with many
basic use cases. One random example: if you're transcoding an audio/video
and somehow altering the timings within lavfi, you have to give the
subtitles.

Having subtitles within libavfilter is not a fancy utopia to give
ourselves a reason to write a bunch of random filters, it actually helps
addressing real limitations in the current model.

What you are suggesting is basically what we already have: the few
subtitles specific API already present in lavc is what you would want out
of it, but it won't solve the core issues.

-- 
Clément B.