[FFmpeg-devel] [PATCH] libavformat/webvttenc: Allow (but discard) additional streams

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> Carl Eugen Hoyos
> 
> Am Mi., 4. Sept. 2019 um 22:43 Uhr schrieb Soft Works
> <softworkz at hotmail.com>:
> >
> >
> > This allows having a video stream as reference stream  when using the
> > segment muxer
> 
> Please add the command line that needs this patch.
> 
> Carl Eugen

The short version would be this:
------------------
Allows mapping a video stream as reference stream for creating vtt segments:
ffmpeg -i INPUT -map 0:3 -c:0 webvtt -map 0:0 -c:v:0 copy -f segment -segment_format webvtt -segment_time 6 -write_empty_segments 1 -y "sub_segment3%d.vtt"
------------------

Is this sufficient? Otherwise, here's the long story:

THE REQUIREMENT
I need to do HLS segmentation for video/audio while additionally creating one or more subtitle playlists and segments. Very simplified it’s like this:

ffmpeg -i INPUT
  -map 0:0 -map 0:2 -c:v:0 copy -c:a:0 copy -f segment -segment_format mpegts -segment_time 6 -y "segment%d.ts"
  -map 0:3 -c:0 webvtt -f segment -segment_format webvtt -segment_time 6 -write_empty_segments 1 -y "sub_segment3%d.vtt"


THE PROBLEM
The mpegts segments are being created in a constant and regular way. But not necessearily in the case of subtitles: When hitting a time range without any subtitle event, the segmenter stops generating subtitle segments, even when write_empty_segments is configured.
It’s not that write_empty_segments would not be working, though: As soon as a new subtitle event reaches the muxer, it catches up by creating the missed empty segments all at once. This might be OK for precreating content for HLS/VOD scenarios but it’s a behavioral bug when you rely on having accurate results while the process is running.

THE CAUSE
That’s rather trivial: When there are no subtitle events, there are no packets arriving at the segment muxer (for vtt) and there’s probably nothiing that the segment muxer could do about it.

A REASONABLE APPROACH?
After some rather ugly ideas I found that the best way to bring some more action into the vtt segmenter might be to map the video stream in addition to the subtitle stream – not with the purpose of outputting the video – just to ensure regular activity in the segment muxer.
Also, the segment muxer has implementation for having a ‚reference_stream‘ and the video stream is supposed to be this anyway.

Remains one problem: The WebVTT muxer (libavformat/webvttenc.c) errors when there’s more than a single stream. ("Exactly one WebVTT stream is needed.) 
And that’s what this commit does: Removed the single-stream limit from the webvtt muxer and simply discard packets from other streams instead of failing.

softworkz