[FFmpeg-devel] [PATCH] avcodec/qsvenc: make QSV encoder encode VAAPI and D3D11 frames directly
Xiang, Haihao
haihao.xiang at intel.com
Thu Jun 9 09:47:42 EEST 2022
On Wed, 2022-06-08 at 11:13 +0000, Soft Works wrote:
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of Xiang,
> > Haihao
> > Sent: Wednesday, June 8, 2022 10:42 AM
> > To: ffmpeg-devel at ffmpeg.org
> > Cc: Wu, Tong1 <tong1.wu at intel.com>; Chen, Wenbin <wenbin.chen at intel.com>
> > Subject: Re: [FFmpeg-devel] [PATCH] avcodec/qsvenc: make QSV encoder encode
> > VAAPI and D3D11 frames directly
> >
> > On Wed, 2022-06-08 at 05:08 +0000, Soft Works wrote:
> > > > -----Original Message-----
> > > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of Tong
> > > > Wu
> > > > Sent: Tuesday, June 7, 2022 11:22 AM
> > > > To: ffmpeg-devel at ffmpeg.org
> > > > Cc: Tong Wu <tong1.wu at intel.com>; Wenbin Chen <wenbin.chen at intel.com>
> > > > Subject: [FFmpeg-devel] [PATCH] avcodec/qsvenc: make QSV encoder encode
> > > > VAAPI
> > > > and D3D11 frames directly
>
> [..]
>
> > > > 2.35.1.windows.2
> > >
> > > Hi,
> > >
> > > thanks for submitting this patch. Though, I'm afraid, but this
> > >
> > > - fundamentally contradicts the logic of ffmpeg's handling of hw
> >
> > acceleration,
> > > hw device and hw frames contexts
> > > - adds code to an encoder, doing things an encoder is not supposed to do-
> >
> > qsv
> > > encoders and decoders have their own context => QSV
> >
> > nvdec and nvenc have CUDA but nvenc can also support D3D11va, it sounds make
> > sense for me to support D3D11va/vaapi in qsvenc too as d3d11va/vaapi are
> > used
> > internally in MediaSDK.
>
> Can you please post a command line showing nvenc working with input
> from a D3D11VA decoder and without using any hwmap/hwupload/hwdownload
> filters?
>
According to the code below, nvenc may accept d3d11 frames directly,
https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/nvenc.c#L46-L72
so the command below should work
$> ffmpeg -y -hwaccel_output_format d3d11 -hwaccel d3d11va -i input.mp4 -c:v
hevc_nvenc out.mp4
>
> >
> > > - is not safe/guaranteed to work always
> > > there are different requirements for QSV than for other other cases
> > > like VAAPI - for example: QSV requires a fixed-size frame pool
> > > and encoders often need a larger frame pool than VAAPI
> > >
> >
> > Encoders in MediaSDK don't need a fixed pool, probably we may relax this
> > limitation in QSV.
>
> Well - I think they do:
>
> Common
> The QSV hw frames context implementation is using a fixed pool size and
> changing that would be anything but trivial.
>
> D3D11
> The decoders are using and allocating a D3D11 array textures which cannot
> be resized. When the surface count is too low, QSV encoding will fail.
>
> VAAPI
> See vaapi_frames_init(): there is support for dynamically sized pools, but
> not as render targets.
>
>
> The actual problem though is not about fixed vs dynamic pool size - it's
> about whether the number of frames will be sufficient for encoding.
> And when try to use the (d3d11va) decoder's output frames and supply them
> to the encoder, they may very likely be insufficient.
>
>
> QSV encoding always requires quite a number of surfaces.
> You might remember all the extra_hw_frames parameters in the sample
> command lines:
>
> https://trac.ffmpeg.org/wiki/Hardware/QuickSync
>
> -------------
>
> Did you ever test this with a D3D11VA decoder on Windows?
> I'm not sure whether this can work at all (feeding decoder frames
> into the encoder) due to the differing bind_flags requirements.
>
> In case you did test that - could you please post the command line?
vaapi on Linux:
$ ffmpeg -y -hwaccel vaapi -hwaccel_output_format vaapi -i input.mp4 -c:v
hevc_qsv out.mp4
d3d11va on Windows:
$ ffmpeg.exe -y -hwaccel d3d11va -hwaccel_output_format d3d11 -i input.mp4 -c:v
h264_qsv -y out.mp4
Thanks
Haihao
More information about the ffmpeg-devel
mailing list