[FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't support P010 format

[Xiang, Haihao]

On Sat, 2022-11-26 at 02:54 +0000, Soft Works wrote:
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> > James Almer
> > Sent: Saturday, November 26, 2022 3:36 AM
> > To: ffmpeg-devel at ffmpeg.org
> > Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't support
> > P010 format
> > 
> > On 11/25/2022 11:31 PM, Soft Works wrote:
> > > 
> > > > -----Original Message-----
> > > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> > > > James Almer
> > > > Sent: Saturday, November 26, 2022 2:01 AM
> > > > To: ffmpeg-devel at ffmpeg.org
> > > > Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't
> > support
> > > > P010 format
> > > > 
> > > > On 11/25/2022 9:58 PM, Soft Works wrote:
> > > > > 
> > > > > > -----Original Message-----
> > > > > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf
> > Of
> > > > > > James Almer
> > > > > > Sent: Saturday, November 26, 2022 12:58 AM
> > > > > > To: ffmpeg-devel at ffmpeg.org
> > > > > > Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't
> > > > support
> > > > > > P010 format
> > > > > > 
> > > > > > On 11/25/2022 8:51 PM, Soft Works wrote:
> > > > > > > 
> > > > > > > > -----Original Message-----
> > > > > > > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf
> > > > Of
> > > > > > > > James Almer
> > > > > > > > Sent: Saturday, November 26, 2022 12:35 AM
> > > > > > > > To: ffmpeg-devel at ffmpeg.org
> > > > > > > > Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't
> > > > > > support
> > > > > > > > P010 format
> > > > > > > > 
> > > > > > > > On 11/25/2022 8:26 PM, Dong, Ruijing wrote:
> > > > > > > > > [AMD Official Use Only - General]
> > > > > > > > > 
> > > > > > > > > Will it make sense to accept P010 input, however encode to
> > h264
> > > > > > > > 8bit?
> > > > > > > > 
> > > > > > > > If it works (the encoder accepts the 10 bit input even if it
> > > > > > encodes
> > > > > > > > it
> > > > > > > > as 8bit), then i don't see why not. I assume it would also be
> > > > > > faster
> > > > > > > > than using swscale to convert said 10bit input to nv12 before
> > > > > > passing
> > > > > > > > that to the encoder.
> > > > > > > > 
> > > > > > > > Removing support for a pixel format as input in an encoder
> > needs
> > > > a
> > > > > > > > reason other than "It's rarely used", more so when it's a
> > single
> > > > > > > > line.
> > > > > > > > It either needs to not work, or somehow get in the way of
> > > > further
> > > > > > > > improvements.
> > > > > > > 
> > > > > > > Oh sorry, I noticed that there was a misunderstanding.
> > > > > > > 
> > > > > > > When I said "It's rarely used", I didn't mean that as a
> > > > > > justification
> > > > > > > for the removal, it was meant as an explanation why none of the
> > > > > > > hwaccels has implemented it.
> > > > > > > 
> > > > > > > softworkz
> > > > > > 
> > > > > > Alright, then i'll repeat my question: Does it work?
> > > > > 
> > > > > No.
> > > > 
> > > > What does this encoder currently do when you feed it p010 input?
> > What
> > > > does it output?
> > > 
> > > An error:
> > > 
> > > 
> > > 1. 10bit from HW context:
> > > 
> > > 
> > > [graph 0 video input from stream 0:0 @ 000001dc301aeec0] w:3840
> > h:2160 pixfmt:yuv420p10le tb:1/60000 fr:60000/1001 sar:1/1
> > > [auto_scale_0 @ 000001dc2362a700] w:iw h:ih flags:'' interl:0
> > > [hwupload at f1 @ 000001dc2944ef00] auto-inserting filter
> > 'auto_scale_0' between the filter 'graph 0 video input from stream
> > 0:0' and the filter 'hwupload at f1'
> > > [auto_scale_0 @ 000001dc2362a700] w:3840 h:2160 fmt:yuv420p10le
> > sar:1/1 -> w:3840 h:2160 fmt:p010le sar:1/1 flags:0x0
> > > [AVHWDeviceContext @ 000001dc444f9a00] D3D11 Init
> > > [AVHWDeviceContext @ 000001dc444fab80] D3D11 Init
> > > [vpp_qsv at f2 @ 000001dc22a3d880] VPP: input is video memory surface
> > > [vpp_qsv at f2 @ 000001dc22a3d880] VPP: output is video memory surface
> > > [auto_scale_0 @ 000001dc2362a700] w:3840 h:2160 fmt:yuv420p10le
> > sar:1/1 -> w:3840 h:2160 fmt:p010le sar:1/1 flags:0x0
> > >      Last message repeated 2 times
> > > [h264_qsv @ 000001dc161b6040] Using input frames context (format
> > qsv) with h264_qsv encoder.
> > > [h264_qsv @ 000001dc161b6040] Encoder: input is video memory
> > surface
> > > [h264_qsv @ 000001dc161b6040] Using the average variable bitrate
> > (AVBR) ratecontrol method
> > > [h264_qsv @ 000001dc161b6040] Current pixel format is unsupported
> > > [h264_qsv @ 000001dc161b6040] some encoding parameters are not
> > supported by the QSV runtime. Please double check the input
> > parameters.
> > > Error initializing output stream 0:0 -- Error while opening encoder
> > for output stream #0:0 - maybe incorrect parameters such as bit_rate,
> > rate, width or height
> > > [AVHWDeviceContext @ 000001dc444f9a00] D3D11 Uninit
> > > [AVIOContext @ 000001dc16197c80] Statistics: 0 bytes written, 0
> > seeks, 0 writeouts
> > > [AVHWDeviceContext @ 000001dc444fab80] D3D11 Uninit
> > > [AVIOContext @ 000001dc161839c0] Statistics: 131146 bytes read, 2
> > seeks
> > > Conversion failed!
> > > 
> > > 
> > > 2. 10bit from SW context:
> > > 
> > > 
> > > [graph 0 video input from stream 0:0 @ 0000019e915dee00] w:3840
> > h:2160 pixfmt:yuv420p10le tb:1/60000 fr:60000/1001 sar:1/1
> > > [auto_scale_0 @ 0000019ee99936c0] w:iw h:ih flags:'' interl:0
> > > [format @ 0000019ee9993240] auto-inserting filter 'auto_scale_0'
> > between the filter 'Parsed_null_0' and the filter 'format'
> > > [auto_scale_0 @ 0000019ee99936c0] w:3840 h:2160 fmt:yuv420p10le
> > sar:1/1 -> w:3840 h:2160 fmt:p010le sar:1/1 flags:0x0
> > >      Last message repeated 3 times
> > > [h264_qsv @ 0000019ee9995dc0] Using device qd1 (type qsv) with
> > h264_qsv encoder.
> > > [h264_qsv @ 0000019ee9995dc0] Encoder: input is system memory
> > surface
> > > [h264_qsv @ 0000019ee9995dc0] Using the average variable bitrate
> > (AVBR) ratecontrol method
> > > [h264_qsv @ 0000019ee9995dc0] Current pixel format is unsupported
> > > [h264_qsv @ 0000019ee9995dc0] some encoding parameters are not
> > supported by the QSV runtime. Please double check the input
> > parameters.
> > > Error initializing output stream 0:0 -- Error while opening encoder
> > for output stream #0:0 - maybe incorrect parameters such as bit_rate,
> > rate, width or height
> > > [AVIOContext @ 0000019ef62b4000] Statistics: 0 bytes written, 0
> > seeks, 0 writeouts
> > > [AVIOContext @ 0000019ee995abc0] Statistics: 131146 bytes read, 2
> > seeks
> > > Conversion failed!
> > > 
> > > softworkz
> > 
> > Alright, thanks for testing it. The commit message should mention the
> > pixel format is being removed as it's unsupported, then.
> 
> The QuickSync architecture does not have any scaling/color conversion
> capabilities at the encoder stage. There's the VPP filtering and they
> also have a decoder-size fixed-function scaling which is separate from
> VPP scaling (similar to CUVID). It's not yet exposed via ffmpeg, though.
> 
> Generally - when other video processing is required, it doesn't make 
> much sense to do encoder-side scaling/conversion. You'll rather
> want to scale down first and then do the other processing on the 
> smaller frames.
> 
> The only case where I've seen an encoder-side scaling feature is 
> MediaCodec, but well - there's no custom processing anyway.
> 

Right, P010 is not supported by h264_qsv encoder, I'll update the commit
message. 

Thanks
Haihao