[FFmpeg-devel] [PATCH v5 24/25] fftools/ffmpeg: Introduce subtitle filtering and new frame-based subtitle encoding

[Andreas Rheinhardt]

Soft Works:
> 
> 
> ________________________________________
> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> on behalf of Andreas Rheinhardt <andreas.rheinhardt at outlook.com>
> Sent: Saturday, June 25, 2022 1:42 PM
> To: ffmpeg-devel at ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH v5 24/25] fftools/ffmpeg: Introduce subtitle filtering and new frame-based subtitle encoding
> 
> Michael Niedermayer:
>> On Sat, Jun 25, 2022 at 09:57:56AM +0000, softworkz wrote:
>>> From: softworkz <softworkz at hotmail.com>
>>>
>>> This commit actually enables subtitle filtering in ffmpeg by
>>> sending and receiving subtitle frames to and from a filtergraph.
>>>
>>> The heartbeat functionality from the previous sub2video implementation
>>> is removed and now provided by the 'subfeed' filter.
>>> The other part of sub2video functionality is retained by
>>> auto-insertion of the new graphicsub2video filter.
>>>
>>> Justification for changed test refs:
>>>
>>> - sub2video
>>>   The previous results were incorrect. The command line for this test
>>>   specifies -r 5 (5 frames per second), which is now fulfilled by the
>>>   additional frames in the reference output.
>>>   Example: The first subtitle time is 499000, the second is 15355000,
>>>   which means 0.5s and 15.35s with a difference of 14.85s.
>>>   15s * 5fps = 75 frames and that's now the exact number of video
>>>   frames between these two subtitle events.
>>>
>>> - sub2video_basic
>>>   The previous results had some incorrect output because multiple
>>>   frames had the same dts
>>>   The non-empty content frames are visually identical, the different
>>>   CRC is due to the different blending algorithm that is being used.
>>>
>>> - sub2video_time_limited
>>>   Subtitle frames are emitted to the filter graphs at a 5 fps rate
>>>   by default. The time limit for this test is 15s * 5fps = 75 frames
>>>   which matches the count in the new ref.
>>>
>>> - sub-dvb
>>>   Running ffprobe -show_frames on the source file shows that there
>>>   are 7 subtitle frames with 0 rects in the source at the start
>>>   and 2 at the end. This translates to the 14 and 4 additional
>>>   entries in the new test results.
>>>
>>> - filter-overlay-dvdsub-2397
>>>   Overlay results have slightly different CRCs due to different
>>>   blending implementation
>>>
>>> - sub-scc
>>>   The first entry is no longer in the output because it is before
>>>   the actual start time and the strim filter removes such entries
>>>   now (like for video and audio)
>>>
>>> Signed-off-by: softworkz <softworkz at hotmail.com>
>>> ---
>>>  fftools/ffmpeg.c                          |  613 +++++-----
>>>  fftools/ffmpeg.h                          |   17 +-
>>>  fftools/ffmpeg_filter.c                   |  270 +++--
>>>  fftools/ffmpeg_hw.c                       |    2 +-
>>>  fftools/ffmpeg_opt.c                      |   28 +-
>>>  tests/ref/fate/filter-overlay-dvdsub-2397 |  182 +--
>>>  tests/ref/fate/sub-dvb                    |  162 +--
>>>  tests/ref/fate/sub-scc                    |    1 -
>>>  tests/ref/fate/sub2video                  | 1091 +++++++++++++++++-
>>>  tests/ref/fate/sub2video_basic            | 1238 +++++++++++++++++++--
>>>  tests/ref/fate/sub2video_time_limited     |   78 +-
>>>  11 files changed, 3010 insertions(+), 672 deletions(-)
>>
>> seems to break fate
>>
>> Input #0, ass, from 'fate-suite//sub/1ededcbd7b.ass':
>>   Duration: N/A, bitrate: N/A
>>   Stream #0:0: Subtitle: ass
>> Codec AVOption threads (set the number of threads) specified for input file #0 (fate-suite//sub/1ededcbd7b.ass) has not been used for any stream. The most likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some decoder which was not actually used for any stream.
>> Codec AVOption thread_type (select multithreading type) specified for input file #0 (fate-suite//sub/1ededcbd7b.ass) has not been used for any stream. The most likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some decoder which was not actually used for any stream.
>> Stream mapping:
>>   Stream #0:0 -> #0:0 (ass (ssa) -> ass (ssa))
>> cur_dts is invalid st:0 (0) [init:0 i_done:0 finish:0] (this is harmless if it occurs once at the start per stream)
>> subtitle input filter: decoding size 1280x720
>> [out_0_0 @ 0x557cab64bbc0] Subtitle format change from 1 to 3
>> Assertion c > 0 failed at libavutil/mathematics.c:61
> 
> That's an av_assert2-assert. softworkz probably didn't test these.
> 
> 
> Yup. To be honest, I wasn't even aware that I'm supposed to. It seems patchwork doesn't do either?
> 
> Do I need a different configure or make fate?
> 

The supported way to set it is to pass --assert-level=2 to configure.
(You can also modify config.h by adding "#define ASSERT_LEVEL 2"; if you
have not used assert-level during configure, you should have no
ASSERT_LEVEL in config.h, but better check this.)

- Andreas