[FFmpeg-devel] [RFC] How to deal with libavfilter buffer overflows?

[Stefano Sabatini]

$ ffmpeg -f lavfi -i "testsrc=d=10, split[t0][t1]; [t0]select='gte(t,5)',setpts=PTS-STARTPTS[s0]; [t1] select='lt(t,5)', setpts=PTS-STARTPTS[s1]; [s0][s1] concat" -y outtest.mp4
[...]
Input #0, lavfi, from 'testsrc=d=10, split[t0][t1]; [t0]select='gte(t,5)',setpts=PTS-STARTPTS[s0]; [t1] select='lt(t,5)', setpts=PTS-STARTPTS[s1]; [s0][s1] concat':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: rawvideo (RGB[24] / 0x18424752), rgb24, 320x240 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 1000k tbn, 1000k tbc
[...]
Output #0, mp4, to 'outtest.mp4':
  Metadata:
    encoder         : Lavf56.9.101
    Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv444p, 320x240 [SAR 1:1 DAR 4:3], q=-1--1, 25 fps, 12800 tbn, 25 tbc
    Metadata:
      encoder         : Lavc56.8.102 libx264
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[out @ 0x3140560] 100 buffers queued in out, something may be wrong.
frame=  125 fps=0.0 q=-1.0 Lsize=      25kB time=00:00:04.92 bitrate=  42.4kbits/s    
video:23kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 9.620967%

I frequently stumble upon situations when buffer overflows happen, and
they are usually a showstopper in many applications. For example when
recombining streams with the concat filter, or when overlaying. The
output is usually playable but misses audio or video frames.  For
example in the case above, the command is trying to switch order of
the two input parts, but only the second part is displayed and the
user is left with the vague message "something may be wrong".

So the current buffering system sometimes fails, and the user has no
clue to understand where the problem lies. I want at least to improve
diagnostics, but I want to listen about other developers experience
first, and if you have ideas to tackle such issues.

I can provide more use cases if required.
-- 
FFmpeg = Foolish and Fanciful Multimedia Powerful Elected Gladiator