[FFmpeg-user] Strange behaviour ffmpeg ver 6.0

[Nicholas Clement]

I am pasting tdarr commands but they were all tested manually in cli with jellyfin and jammy's ffmpeg deb file.
2 thigs:

  1.  I can't get higher than 15-20% of util on the GPU.
Even if I run 10 concurent transcoding or 1.
  2.  The transcoding fps drops from 350 ish to 50 ish within 10-15 minutes.
What's weird here is once it gets to the 50 ish numbers, if I start other transcoding jobs, the fps is not affected.

Temperatures are normal, network is 10G.
I have tested transcode on local nvme. Same issue
This happens on all GPUs.
PSU are server graded 1200W
I'm not sure what logs or others infos you need to help me troubleshoot this.

tdarr-ffmpeg -c:v h264_cuvid -i "/media/movies/1922 (2017) [Unknown] [NR] [voted 6.5] [Horror]/1922 (2017) [Unknown] [NR] [voted 6.5] [Horror].mkv" -map 0 -c:v hevc_nvenc -cq:v 19 -b:v 3857k -minrate 2699k -maxrate 5014k -bufsize 7715k -spatial_aq:v 1 -rc-lookahead:v 32 -c:a copy -c:s copy -max_muxing_queue_size 9999 -pix_fmt p010le "/temp/1922 (2017) [Unknown] [NR] [voted 6.5] [Horror]-TdarrCacheFile-XUoEn9krtp.mkv"

ffmpeg --version
ffmpeg version 6.0-Jellyfin Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libsvtav1 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-opencl --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100

nvidia-smi
Tue Oct 10 15:18:39 2023
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.104.12             Driver Version: 535.104.12   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 4070        On  | 00000000:00:10.0 Off |                  N/A |
| 32%   48C    P2              31W / 200W |    751MiB / 12282MiB |     12%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
|   1  Quadro RTX 4000                On  | 00000000:00:1B.0 Off |                  N/A |
| 36%   63C    P0              46W / 125W |    628MiB /  8192MiB |     14%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
|   2  Quadro RTX 4000                On  | 00000000:00:1C.0 Off |                  N/A |
| 37%   64C    P0              38W / 125W |    628MiB /  8192MiB |     11%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A   1720349      C   tdarr-ffmpeg                                740MiB |
|    1   N/A  N/A   1720486      C   tdarr-ffmpeg                                624MiB |
|    2   N/A  N/A   1720690      C   tdarr-ffmpeg                                624MiB |
+---------------------------------------------------------------------------------------+