[FFmpeg-user] amd hwaccel encoding + overlay filter is not working together
Chen, Wenbin
wenbin.chen at intel.com
Fri Mar 29 04:51:53 EET 2024
> I just successfully tested how to use AMD GPU's hwaccel to re-encode a file
> with ffmpeg. Using this command:
> ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128
> -hwaccel_output_format vaapi -i input.mp4 -c:v hevc_vaapi -c:a copy -crf 23
> output.mp4
> encoding is much faster.
> Testing it on a sample file and wrapping around with time:
> time ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128
> -hwaccel_output_format vaapi -i input.mp4 -c:v hevc_vaapi -c:a copy -crf 23
> -t 60 output.mp4
> frame= 1800 fps= 35 q=-0.0 Lsize= 294431kB time=00:00:59.98
> bitrate=40209.7kbits/s dup=2 drop=0 speed=1.16x
>
> real 0m52.024s
> user 0m2.304s
> sys 0m10.866s
> ^ This is definitely using the GPU and hwaccel. During the encoding, I see
> no cpu usage spike in top, or increase of loadavg, etc.
>
> While the "classic" re-encode for the same file:
> time ffmpeg -i input.mp4 -c:v libx265 -c:a copy -crf 23 -t 60 output.mp4
> ...
>
> encoded 1800 frames in 460.63s (3.91 fps), 27046.56 kb/s, Avg QP:27.49
>
> real 7m41.169s
> user 109m18.008s
> sys 0m5.764s
>
> This is definitely using the cpu. During this 7 minutes and 41 seconds, the
> all the cpu cores were 100% used, load was around the number of cpu cores,
> etc.
>
> The actual command I try to execute:
> ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128
> -hwaccel_output_format vaapi -i input.mp4 -r 1 -i overlay%04d.png
> -filter_complex overlay -r 7.5 -an -c:v hevc_vaapi -crf 22 output.mp4
> But it's not working.
> I've got this output:
> ---------<snip>--------
> $ ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128
> -hwaccel_output_format vaapi -i input.mp4 -r 1 -i overlay%04d.png
> -filter_complex overlay -r 7.5 -an -c:v hevc_vaapi -crf 22 output.mp4
> ffmpeg version 6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
> built with FreeBSD clang version 16.0.6 (
> https://github.com/llvm/llvm-project.git llvmorg-16.0.6-0-g7cbf1a259152)
> configuration: --prefix=/usr/local --mandir=/usr/local/man
> --datadir=/usr/local/share/ffmpeg --docdir=/usr/local/share/doc/ffmpeg
> --pkgconfigdir=/usr/local/libdata/pkgconfig --disable-static
> --disable-libcelt --enable-shared --enable-pic --enable-gpl --cc=cc
> --cxx=c++ --disable-alsa --disable-libopencore-amrnb
> --disable-libopencore-amrwb --enable-libaom --disable-libaribb24
> --disable-libaribcaption --enable-asm --enable-libass --disable-libbs2b
> --disable-libcaca --disable-libcdio --disable-libcodec2 --enable-libdav1d
> --disable-libdavs2 --disable-libdc1394 --disable-debug --enable-htmlpages
> --enable-libdrm --disable-libfdk-aac --disable-libflite --enable-fontconfig
> --enable-libfreetype --enable-frei0r --disable-libfribidi --disable-gcrypt
> --disable-libglslang --disable-libgme --enable-gmp --enable-gnutls
> --enable-version3 --disable-libgsm --enable-libharfbuzz --enable-iconv
> --disable-libilbc --disable-libjack --enable-libjxl --disable-libklvanc
> --disable-libkvazaar --disable-ladspa --enable-libmp3lame --enable-lcms2
> --disable-liblensfun --disable-libbluray --enable-libplacebo
> --disable-librsvg --disable-librtmp --enable-libxml2 --disable-lv2
> --disable-mbedtls --disable-libmfx --disable-libmodplug --disable-libmysofa
> --enable-network --disable-nonfree --enable-nvenc --disable-openal
> --disable-opencl --disable-opengl --disable-libopenh264
> --disable-libopenjpeg --disable-libopenmpt --disable-openssl
> --disable-libopenvino --enable-optimizations --enable-libopus
> --disable-pocketsphinx --disable-libpulse --disable-librabbitmq
> --disable-librav1e --disable-librist --enable-runtime-cpudetect
> --disable-librubberband --disable-sdl2 --enable-libshaderc
> --disable-libsmbclient --disable-libsnappy --disable-sndio
> --disable-libsoxr --disable-libspeex --disable-libsrt --disable-libssh
> --enable-libsvtav1 --disable-libtensorflow --disable-libtesseract
> --disable-libtheora --disable-libtwolame --disable-libuavs3d
> --enable-libv4l2 --enable-vaapi --disable-vapoursynth --enable-vdpau
> --disable-libvidstab --enable-libvmaf --enable-libvorbis
> --disable-libvo-amrwbenc --disable-libvpl --enable-libvpx --enable-vulkan
> --enable-libwebp --enable-libx264 --enable-libx265 --disable-libxavs2
> --enable-libxcb --disable-libxvid --disable-outdev=xv --disable-libzimg
> --disable-libzmq --disable-libzvbi
> libavutil 58. 29.100 / 58. 29.100
> libavcodec 60. 31.102 / 60. 31.102
> libavformat 60. 16.100 / 60. 16.100
> libavdevice 60. 3.100 / 60. 3.100
> libavfilter 9. 12.100 / 9. 12.100
> libswscale 7. 5.100 / 7. 5.100
> libswresample 4. 12.100 / 4. 12.100
> libpostproc 57. 3.100 / 57. 3.100
> Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'input.mp4':
> Metadata:
> major_brand : isom
> minor_version : 512
> compatible_brands: isomiso2avc1mp41
> creation_time : 2023-09-03T23:12:04.000000Z
> encoder : Lavf60.3.100
> Duration: 00:01:15.01, start: 0.000000, bitrate: 14849 kb/s
> Stream #0:0[0x1](eng): Video: h264 (High) (avc1 / 0x31637661),
> yuv420p(progressive), 1920x1080, 14750 kb/s, 30 fps, 30 tbr, 60k tbn
> (default)
> Metadata:
> creation_time : 2023-09-03T23:12:04.000000Z
> handler_name : VideoHandler
> vendor_id : [0][0][0][0]
> encoder : h264
> Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 32000 Hz,
> mono, fltp, 96 kb/s (default)
> Metadata:
> creation_time : 2023-09-03T23:12:04.000000Z
> handler_name : SoundHandler
> vendor_id : [0][0][0][0]
> Input #1, image2, from 'overlay%04d.png':
> Duration: 00:00:03.00, start: 0.000000, bitrate: N/A
> Stream #1:0: Video: png, rgba(pc, gbr/unknown/unknown), 1920x1080, 25
> fps, 25 tbr, 25 tbn
> amdgpu: os_same_file_description couldn't determine if two DRM fds
> reference the same file description.
> If they do, bad things may happen!
> [out#0/mp4 @ 0x1e333e6b5600] Codec AVOption crf (Select the quality for
> constant quality mode) has not been used for any stream. The most likely
> reason is either wrong type (e.g. a video option with no video streams) or
> that it is a private option of some encoder which was not actually used for
> any stream.
> Stream mapping:
> Stream #0:0 (h264) -> overlay
> Stream #1:0 (png) -> overlay
> overlay:default -> Stream #0:0 (hevc_vaapi)
> Press [q] to stop, [?] for help
> [in#1/image2 @ 0x1e333e62c600] Thread message queue blocking; consider
> raising the thread_queue_size option (current value: 8)
> Impossible to convert between the formats supported by the filter 'graph 0
> input from stream 0:0' and the filter 'auto_scale_0'
> [fc#0 @ 0x1e333e66a080] Error reinitializing filters!
> Failed to inject frame into filter network: Function not implemented
> Error while filtering: Function not implemented
> Impossible to convert between the formats supported by the filter 'graph 0
> input from stream 0:0' and the filter 'auto_scale_0'
> [fc#0 @ 0x1e333e66a080] Error reinitializing filters!
> Failed to inject frame into filter network: Function not implemented
> [out#0/mp4 @ 0x1e333e6b5600] Nothing was written into output file,
> because
> at least one of its streams received no packets.
> frame= 0 fps=0.0 q=0.0 Lsize= 0kB time=N/A bitrate=N/A speed=N/A
>
> Conversion failed!
> ---------<snap>--------
> I'm not sure, if it's completely impossible, because some feature is
> missing on ffmpeg, or from the dri driver, or from freebsd (though, the
> simple task, to re-encode an mp4 file was working on this same host), or I
> just need to add some extra filter, so the output of the overlay filter
> would be compatible with the encoder / vaapi's input.
> Though, based on the error message, I don't even know if it's the input of
> the overlay filter, or the output of the overlay filter is problematic.
> Can someone please help?
> By googling the error message, I found this, but I'm not sure how to apply
> this to my filter, or if it's even the same problem.
> https://superuser.com/questions/1633883/ffmpeg-hevc-vaapi-impossible-to-
> convert-between-the-formats-supported-by-the-fi
> Based on the answers there, I tried to convert it further, but without any
> luck:
> ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128
> -hwaccel_output_format vaapi -i input.mp4 -r 1 -i overlay%04d.png
> -filter_complex overlay -r 7.5 -an -c:v hevc_vaapi -crf 22 output.mp4
> ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128
> -hwaccel_output_format vaapi -i input.mp4 -r 1 -i overlay%04d.png
> -filter_complex 'overlay,format=nv12|vaapi,hwupload' -r 7.5 -an -c:v
> hevc_vaapi -crf 22 output.mp4
> ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128
> -hwaccel_output_format vaapi -i input.mp4 -r 1 -i overlay%04d.png
> -filter_complex 'overlay,format=yuv420p|vaapi,hwupload' -r 7.5 -an -c:v
> hevc_vaapi -crf 22 output.mp4
> ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128
> -hwaccel_output_format vaapi -i input.mp4 -r 1 -i overlay%04d.png
> -filter_complex 'overlay,hwupload' -r 7.5 -an -c:v hevc_vaapi -crf 22
> output.mp4
> ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128
> -hwaccel_output_format vaapi -i input.mp4 -r 1 -i overlay%04d.png
> -filter_complex 'overlay,format=nv12|vaapi,hwupload' -r 7.5 -an -c:v
> hevc_vaapi -crf 22 output.mp4
> ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128
> -hwaccel_output_format vaapi -i input.mp4 -r 1 -i overlay%04d.png
> -filter_complex 'overlay,format=yuv420p|vaapi,hwupload' -r 7.5 -an -c:v
> hevc_vaapi -crf 22 output.mp4
> ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128
> -hwaccel_output_format vaapi -i input.mp4 -r 1 -i overlay%04d.png
> -filter_complex 'overlay,format=yuv420p,hwupload' -r 7.5 -an -c:v
> hevc_vaapi -crf 22 output.mp4
>
> Thank you for every help!
> Gyu
The command should be changed to like this:
ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128
-hwaccel_output_format vaapi -i input.mp4 -r 1 -i overlay%04d.png
-filter_complex "[1:v]hwupload,format=vaapi[1v];[0:v][1v]overlay_vaapi"
-r 7.5 -an -c:v hevc_vaapi -crf 22 output.mp4
When you use "-hwaccel_output_format vaapi", it means the output frames
are sotred in device memory. Classic overlay filter cannot handle frames in device
memory, so you need to use overlay_vaapi.
png images are decoded to system memory, so for the same reason, you need to upload
them to device memory.
- Wenbin
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
More information about the ffmpeg-user
mailing list