[FFmpeg-devel] Add a frame/buffer pool to AVFilter

[Matthieu Bouron]

---

Hello,

The following two commits aim to add a video frame pool to AVFilter.
This feature was originally introduced by commit
39f66edbeae5ccabefe38b2fcb25d6c242d868c0 but was removed later on when the
AVFilter API was changed to use AVFrame.

The AVVideoFramePool implementation is similar to the FramePool implementation
from libavcodec (except it's only for video), meaning that the pool keeps references
on the underlying buffers rather than the avframes themselves. The pool keeps
reference to "frames" with the same size, format and alignement.

On the AVFilter side, I choose to put a video frame pool per AVFilterLink to keep it
simple and more clear (to me at least) (and avoid looping on a list of frames of
different sizes, like it was done before). This is indeed at the cost of potentially
using more memory depending on the scenario.

Having a video frame pool avoid some malloc/free calls but also benefits a lot when
writing to memory on arm devices (see benchmark below).

Here are some benchmark performed on a BeagleBone Black board:

Without the video frame pool:

./ffmpeg -f lavfi -i "testsrc2=1920x1080,format=nv12,format=rgba" -f null -t 1 -
nvx->rgbx took 24615.000000
nvx->rgbx took 24617.000000
nvx->rgbx took 24601.000000
nvx->rgbx took 24675.000000
nvx->rgbx took 25723.000000
nvx->rgbx took 24535.000000
nvx->rgbx took 24592.000000
nvx->rgbx took 24637.000000
nvx->rgbx took 24567.000000
nvx->rgbx took 24641.000000
nvx->rgbx took 25521.000000
nvx->rgbx took 24670.000000
nvx->rgbx took 24598.000000
nvx->rgbx took 24547.000000
nvx->rgbx took 24644.000000
nvx->rgbx took 25414.000000
nvx->rgbx took 25099.000000
nvx->rgbx took 25146.000000
nvx->rgbx took 25447.000000
nvx->rgbx took 25094.000000
nvx->rgbx took 25436.000000
nvx->rgbx took 25393.000000
nvx->rgbx took 25411.000000
nvx->rgbx took 25124.000000
nvx->rgbx took 25139.000000
frame=   25 fps=8.4 q=-0.0 Lsize=N/A time=00:00:01.00 bitrate=N/A

With the video frame pool:

./ffmpeg -f lavfi -i "testsrc2=1920x1080,format=nv12,format=rgba" -f null -t 1 -
nvx->rgbx took 10430.000000
nvx->rgbx took 10476.000000
nvx->rgbx took 10428.000000
nvx->rgbx took 10462.000000
nvx->rgbx took 11090.000000
nvx->rgbx took 11777.000000
nvx->rgbx took 10845.000000
nvx->rgbx took 10924.000000
nvx->rgbx took 10840.000000
nvx->rgbx took 10921.000000
nvx->rgbx took 10892.000000
nvx->rgbx took 10927.000000
nvx->rgbx took 10857.000000
nvx->rgbx took 10900.000000
nvx->rgbx took 10878.000000
nvx->rgbx took 10745.000000
nvx->rgbx took 10935.000000
nvx->rgbx took 10433.000000
nvx->rgbx took 10470.000000
nvx->rgbx took 10412.000000
nvx->rgbx took 10425.000000
nvx->rgbx took 10429.000000
nvx->rgbx took 10832.000000
nvx->rgbx took 10413.000000
nvx->rgbx took 10423.000000
frame=   25 fps= 12 q=-0.0 Lsize=N/A time=00:00:01.00 bitrate=N/A

Best regards,
Matthieu