[FFmpeg-user] Problems doing mosaics with ffmpeg

Mon May 12 15:39:07 CEST 2014

Hello all,

I´ve been playing for a while with ffmpeg and the tools for creating mosaics
from several file or multicast (udp) inputs.

While with file inputs (more than 4) seems stable though it has pixelations
more or less alwais at the same time with multicast inputs I can´t get more
than 4 video inputs mosaic before a I get circular buffers overflow and run
out of RAM memory until the system gets dead.

An example of an stable mosaic without noticeable drops or memory running
out is the next:

ffmpeg -i
'udp://224.168.223.61:2061?buffer_size=1000000?fifo_size=2000000&overrun_non
fatal=1' -i
'udp://224.168.223.45:2045?buffer_size=1000000?fifo_size=2000000&overrun_non
fatal=1' -i
'udp://224.168.223.44:2044?buffer_size=1000000?fifo_size=2000000&overrun_non
fatal=1' -i
'udp://224.168.223.62:2062?buffer_size=1000000?fifo_size=2000000&overrun_non
fatal=1' \

-threads 0 \

-max_alloc 5000000000 \

-filter_complex "nullsrc=s=720x576 [base]; \

[0:v]
drawtext=fontfile=/usr/share/fonts/truetype/ttf-dejavu/DejaVuSans.ttf:text=D
iscovery Channel
SD:fontcolor=white:fontsize=34:borderw=3:x=(w-tw)/2:y=(th-16):box=0:boxcolor
=0x000000, setpts=PTS-STARTPTS, scale=360:288 [upperleft]; \

\

[1:v]
drawtext=fontfile=/usr/share/fonts/truetype/ttf-dejavu/DejaVuSans.ttf:text=F
oxLife Turkey
HD:r=25:fontcolor=white:fontsize=80:borderw=6:x=(w-tw)/2:y=(th-40):box=0:box
color=0x000000, setpts=PTS-STARTPTS, scale=360:288 [upperright]; \

\

[2:v]
drawtext=fontfile=/usr/share/fonts/truetype/ttf-dejavu/DejaVuSans.ttf:text=N
atGeo Wild
HD:r=25:fontcolor=white:fontsize=80:borderw=6:x=(w-tw)/2:y=(th-32):box=0:box
color=0x000000, setpts=PTS-STARTPTS, scale=360:288 [lowerleft]; \

\

[3:v]
drawtext=fontfile=/usr/share/fonts/truetype/ttf-dejavu/DejaVuSans.ttf:text=F
X Portugal
HD:r=25:fontcolor=white:fontsize=80:borderw=6:x=(w-tw)/2:y=(th-40):box=0:box
color=0x000000, setpts=PTS-STARTPTS, scale=360:288 [lowerright]; \

\

[base][upperleft] overlay=shortest=0:x=0 [tmp1]; \

[tmp1][upperright] overlay=shortest=0:x=360 [tmp2]; \

[tmp2][lowerleft] overlay=shortest=0:y=288 [tmp3]; \

[tmp3][lowerright] overlay=shortest=0:x=360:y=288" -c:v libx264 -r 25 -crf
23 -aspect 16:9 -pix_fmt yuv420p -b:v 6M -c:a libfdk_aac -b:a 64 -ac 2 -f
mpegts -muxrate 7M "udp://224.168.223.201:2201?packet_size=1316"

As long as you add more inputs, the circular buffer overun appears though
you change the values of buffer_size and fifo_size. Setting only fifo_size
can help but for avoiding the memory leackage buffer_size helps. 

This mosaic can be done using pad and also works well with 4 inputs with
more resolution:

ffmpeg -i
'udp://224.168.223.44:2044?buffer_size=1000000?fifo_size=3000000&overrun_non
fatal=1' -i
'udp://224.168.223.44:2044?buffer_size=1000000?fifo_size=3000000&overrun_non
fatal=1' -i
'udp://224.168.223.44:2044?buffer_size=1000000?fifo_size=3000000&overrun_non
fatal=1' -i
'udp://224.168.223.44:2044?buffer_size=1000000?fifo_size=3000000&overrun_non
fatal=1'  \

-threads 0 \

-max_alloc 5000000000 \

-filter_complex \

\

"[0:v]
drawtext=fontfile=/usr/share/fonts/truetype/ttf-dejavu/DejaVuSerif-Bold.ttf:
text=CANAL
1:fontcolor=white:fontsize=75:x=(w-tw)/2:y=th-20:box=1:boxcolor=0x000000,
setpts=PTS-STARTPTS, scale=720:576 [video_0_final]; \

\

[1:v]
drawtext=fontfile=/usr/share/fonts/truetype/ttf-dejavu/DejaVuSerif-Bold.ttf:
text=CANAL
2:fontcolor=white:fontsize=75:x=(w-tw)/2:y=th-20:box=1:boxcolor=0x000000,
setpts=PTS-STARTPTS, scale=720:576 [video_1_final]; \

\

[2:v]
drawtext=fontfile=/usr/share/fonts/truetype/ttf-dejavu/DejaVuSerif-Bold.ttf:
text=CANAL
3:fontcolor=white:fontsize=75:x=(w-tw)/2:y=th-20:box=1:boxcolor=0x000000,
setpts=PTS-STARTPTS, scale=720:576 [video_2_final]; \

\

[3:v]
drawtext=fontfile=/usr/share/fonts/truetype/ttf-dejavu/DejaVuSerif-Bold.ttf:
text=CANAL
4:fontcolor=white:fontsize=35:x=(w-tw)/2:y=th:box=1:boxcolor=0x000000,
setpts=PTS-STARTPTS, scale=720:576 [video_3_final]; \

\

[video_0_final]pad=iw*2:ih*2 [temp1]; \

[temp1][video_1_final] overlay=W/2 [temp2]; \

[temp2][video_2_final] overlay=W/512:H/2 [temp3]; \

[temp3][video_3_final] overlay=W/2:H/2 [fileout]" -map "[fileout]" -vcodec
libx264 -crf 23 -s 720x576 -pix_fmt yuv420p -b:v 8M -maxrate:v 8M -minrate:v
8M -bufsize 8M -c:a libfdk_aac -b:a 48k -ac 2 -f mpegts -f mpegts -muxrate
9M -f mpegts "udp://224.168.223.201:2201"

The next example, is a 3x3 mosaic. Similar the one above but with this one,
I get pixelations and freezed very often and also it runs out of memory and
finally got circular buffer overrun though you play a little bit with fifo
and buffer size. When I use file input, it can be without buffer overrun but
you can see pixelations and freezes as well.

ffmpeg -i
'udp://224.168.223.44:2044?fifo_size=1000000?buffer_size=2000000&overrun_non
fatal=1' -i
'udp://224.168.223.44:2044?fifo_size=1000000?buffer_size=2000000&overrun_non
fatal=1' -i
'udp://224.168.223.44:2044?fifo_size=1000000?buffer_size=2000000&overrun_non
fatal=1' -i
'udp://224.168.223.44:2044?fifo_size=1000000?buffer_size=2000000&overrun_non
fatal=1' -i
'udp://224.168.223.44:2044?fifo_size=1000000?buffer_size=2000000&overrun_non
fatal=1' -i
'udp://224.168.223.44:2044?fifo_size=1000000?buffer_size=2000000&overrun_non
fatal=1' -i
'udp://224.168.223.44:2044?fifo_size=1000000?buffer_size=2000000&overrun_non
fatal=1' -i
'udp://224.168.223.44:2044?fifo_size=1000000?buffer_size=2000000&overrun_non
fatal=1' -i
'udp://224.168.223.44:2044?fifo_size=1000000?buffer_size=2000000&overrun_non
fatal=1' \

-threads 0 \

-max_alloc 5500000000 \

-filter_complex "nullsrc=s=1920x1080 [base]; \

\

[0:v]
drawtext=fontfile=/usr/share/fonts/truetype/ttf-dejavu/DejaVuSerif-Bold.ttf:
text=CANAL
1:fontcolor=white:fontsize=40:x=(w-tw)/2:y=th-20:box=1:boxcolor=0x000000,
setpts=PTS-STARTPTS, scale=640x360 [1_1]; \

\

[1:v]
drawtext=fontfile=/usr/share/fonts/truetype/ttf-dejavu/DejaVuSerif-Bold.ttf:
text=CANAL
2:fontcolor=white:fontsize=40:x=(w-tw)/2:y=th-20:box=1:boxcolor=0x000000,
setpts=PTS-STARTPTS, scale=640x360 [1_2]; \

\

[2:v]
drawtext=fontfile=/usr/share/fonts/truetype/ttf-dejavu/DejaVuSerif-Bold.ttf:
text=CANAL
3:fontcolor=white:fontsize=40:x=(w-tw)/2:y=th-20:box=1:boxcolor=0x000000,
setpts=PTS-STARTPTS, scale=640x360 [1_3]; \

\

[3:v]
drawtext=fontfile=/usr/share/fonts/truetype/ttf-dejavu/DejaVuSerif-Bold.ttf:
text=CANAL
4:fontcolor=white:fontsize=80:x=(w-tw)/2:y=th-20:box=1:boxcolor=0x000000,
setpts=PTS-STARTPTS, scale=640x360 [2_1]; \

\

[4:v]
drawtext=fontfile=/usr/share/fonts/truetype/ttf-dejavu/DejaVuSerif-Bold.ttf:
text=CANAL
5:fontcolor=white:fontsize=40:x=(w-tw)/2:y=th-20:box=1:boxcolor=0x000000,
setpts=PTS-STARTPTS, scale=640x360 [2_2]; \

\

[5:v]
drawtext=fontfile=/usr/share/fonts/truetype/ttf-dejavu/DejaVuSerif-Bold.ttf:
text=CANAL
6:fontcolor=white:fontsize=40:x=(w-tw)/2:y=th-20:box=1:boxcolor=0x000000,
setpts=PTS-STARTPTS, scale=640x360 [2_3]; \

\

[6:v]
drawtext=fontfile=/usr/share/fonts/truetype/ttf-dejavu/DejaVuSerif-Bold.ttf:
text=CANAL
7:fontcolor=white:fontsize=40:x=(w-tw)/2:y=th-20:box=1:boxcolor=0x000000,
setpts=PTS-STARTPTS, scale=640x360 [3_1]; \

\

[7:v]
drawtext=fontfile=/usr/share/fonts/truetype/ttf-dejavu/DejaVuSerif-Bold.ttf:
text=CANAL
8:fontcolor=white:fontsize=80:x=(w-tw)/2:y=th-20:box=1:boxcolor=0x000000,
setpts=PTS-STARTPTS, scale=640x360 [3_2]; \

\

[8:v]
drawtext=fontfile=/usr/share/fonts/truetype/ttf-dejavu/DejaVuSerif-Bold.ttf:
text=CANAL
9:fontcolor=white:fontsize=80:x=(w-tw)/2:y=th-20:box=1:boxcolor=0x000000,
setpts=PTS-STARTPTS, scale=640x360 [3_3]; \

\

[base][1_1] overlay=shortest=0:x=0:y=0 [tmp1]; \

[tmp1][1_2] overlay=shortest=0:x=640:y=0 [tmp2]; \

[tmp2][1_3] overlay=shortest=0:x=1280:y=0 [tmp3]; \

[tmp3][2_1] overlay=shortest=0:x=0:y=360 [tmp4]; \

[tmp4][2_2] overlay=shortest=0:x=640:y=360 [tmp5]; \

[tmp5][2_3] overlay=shortest=0:x=1280:y=360 [tmp6]; \

[tmp6][3_1] overlay=shortest=0:x=0:y=720 [tmp7]; \

[tmp7][3_2] overlay=shortest=0:x=640:y=720 [tmp8]; \

[tmp8][3_3] overlay=shortest=0:x=1280:y=720" -c:v libx264 -r 45 -crf 23
-aspect 16:9 -pix_fmt yuv420p -b:v 8M -maxrate 8M -minrate 8M -bufsize 8M
-c:a libfdk_aac -b:a 48k -ac 2 -f mpegts -mpegts_flags resend_headers -flags
cgop -sc_threshold 1000000000 -f mpegts -muxrate 11M
"udp://224.168.223.201:2201?ttl=24&packet_size=1316"

I´ve also tested this mosaics with file input instead of udp but happens the
same. Notice that in the las mosaic, I´ve to increase –r to 45 to get at the
output at least 24-25 for avoding the slow motion effect.

The machine is a 16core Xeon proccessor with 6Gb of RAM. The 2x2 mosaic
takes around 20-30% of CPU average and the system has almost 80-85% of free
RAM. The 3x3 takes around 65% of CPU and starts from 65% of free RAM. Using
file input the use of ram changes as it takes more RAM for (I guess)
buffering the file and has a constant output.

Does anyone faced something similar or maybe knows how to fix this circular
buffers overrun or having the machine running out of memmory?

Thanks a lot!

Raúl