My colleagues recently optimized some initial parameter settings for nvenc extension, it improves the PSNR result by approximately 0.5dB. The main idea is assign some optimized initialQPs. I'll send the patch after integrate his improvement to nvenc.c