[FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't support P010 format

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> James Almer
> Sent: Saturday, November 26, 2022 3:36 AM
> To: ffmpeg-devel at ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't support
> P010 format
> 
> On 11/25/2022 11:31 PM, Soft Works wrote:
> >
> >
> >> -----Original Message-----
> >> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> >> James Almer
> >> Sent: Saturday, November 26, 2022 2:01 AM
> >> To: ffmpeg-devel at ffmpeg.org
> >> Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't
> support
> >> P010 format
> >>
> >> On 11/25/2022 9:58 PM, Soft Works wrote:
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf
> Of
> >>>> James Almer
> >>>> Sent: Saturday, November 26, 2022 12:58 AM
> >>>> To: ffmpeg-devel at ffmpeg.org
> >>>> Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't
> >> support
> >>>> P010 format
> >>>>
> >>>> On 11/25/2022 8:51 PM, Soft Works wrote:
> >>>>>
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf
> >> Of
> >>>>>> James Almer
> >>>>>> Sent: Saturday, November 26, 2022 12:35 AM
> >>>>>> To: ffmpeg-devel at ffmpeg.org
> >>>>>> Subject: Re: [FFmpeg-devel] [PATCH] lavc/qsvenc_h264: don't
> >>>> support
> >>>>>> P010 format
> >>>>>>
> >>>>>> On 11/25/2022 8:26 PM, Dong, Ruijing wrote:
> >>>>>>> [AMD Official Use Only - General]
> >>>>>>>
> >>>>>>> Will it make sense to accept P010 input, however encode to
> h264
> >>>>>> 8bit?
> >>>>>>
> >>>>>> If it works (the encoder accepts the 10 bit input even if it
> >>>> encodes
> >>>>>> it
> >>>>>> as 8bit), then i don't see why not. I assume it would also be
> >>>> faster
> >>>>>> than using swscale to convert said 10bit input to nv12 before
> >>>> passing
> >>>>>> that to the encoder.
> >>>>>>
> >>>>>> Removing support for a pixel format as input in an encoder
> needs
> >> a
> >>>>>> reason other than "It's rarely used", more so when it's a
> single
> >>>>>> line.
> >>>>>> It either needs to not work, or somehow get in the way of
> >> further
> >>>>>> improvements.
> >>>>>
> >>>>> Oh sorry, I noticed that there was a misunderstanding.
> >>>>>
> >>>>> When I said "It's rarely used", I didn't mean that as a
> >>>> justification
> >>>>> for the removal, it was meant as an explanation why none of the
> >>>>> hwaccels has implemented it.
> >>>>>
> >>>>> softworkz
> >>>>
> >>>> Alright, then i'll repeat my question: Does it work?
> >>>
> >>> No.
> >>
> >> What does this encoder currently do when you feed it p010 input?
> What
> >> does it output?
> >
> > An error:
> >
> >
> > 1. 10bit from HW context:
> >
> >
> > [graph 0 video input from stream 0:0 @ 000001dc301aeec0] w:3840
> h:2160 pixfmt:yuv420p10le tb:1/60000 fr:60000/1001 sar:1/1
> > [auto_scale_0 @ 000001dc2362a700] w:iw h:ih flags:'' interl:0
> > [hwupload at f1 @ 000001dc2944ef00] auto-inserting filter
> 'auto_scale_0' between the filter 'graph 0 video input from stream
> 0:0' and the filter 'hwupload at f1'
> > [auto_scale_0 @ 000001dc2362a700] w:3840 h:2160 fmt:yuv420p10le
> sar:1/1 -> w:3840 h:2160 fmt:p010le sar:1/1 flags:0x0
> > [AVHWDeviceContext @ 000001dc444f9a00] D3D11 Init
> > [AVHWDeviceContext @ 000001dc444fab80] D3D11 Init
> > [vpp_qsv at f2 @ 000001dc22a3d880] VPP: input is video memory surface
> > [vpp_qsv at f2 @ 000001dc22a3d880] VPP: output is video memory surface
> > [auto_scale_0 @ 000001dc2362a700] w:3840 h:2160 fmt:yuv420p10le
> sar:1/1 -> w:3840 h:2160 fmt:p010le sar:1/1 flags:0x0
> >      Last message repeated 2 times
> > [h264_qsv @ 000001dc161b6040] Using input frames context (format
> qsv) with h264_qsv encoder.
> > [h264_qsv @ 000001dc161b6040] Encoder: input is video memory
> surface
> > [h264_qsv @ 000001dc161b6040] Using the average variable bitrate
> (AVBR) ratecontrol method
> > [h264_qsv @ 000001dc161b6040] Current pixel format is unsupported
> > [h264_qsv @ 000001dc161b6040] some encoding parameters are not
> supported by the QSV runtime. Please double check the input
> parameters.
> > Error initializing output stream 0:0 -- Error while opening encoder
> for output stream #0:0 - maybe incorrect parameters such as bit_rate,
> rate, width or height
> > [AVHWDeviceContext @ 000001dc444f9a00] D3D11 Uninit
> > [AVIOContext @ 000001dc16197c80] Statistics: 0 bytes written, 0
> seeks, 0 writeouts
> > [AVHWDeviceContext @ 000001dc444fab80] D3D11 Uninit
> > [AVIOContext @ 000001dc161839c0] Statistics: 131146 bytes read, 2
> seeks
> > Conversion failed!
> >
> >
> > 2. 10bit from SW context:
> >
> >
> > [graph 0 video input from stream 0:0 @ 0000019e915dee00] w:3840
> h:2160 pixfmt:yuv420p10le tb:1/60000 fr:60000/1001 sar:1/1
> > [auto_scale_0 @ 0000019ee99936c0] w:iw h:ih flags:'' interl:0
> > [format @ 0000019ee9993240] auto-inserting filter 'auto_scale_0'
> between the filter 'Parsed_null_0' and the filter 'format'
> > [auto_scale_0 @ 0000019ee99936c0] w:3840 h:2160 fmt:yuv420p10le
> sar:1/1 -> w:3840 h:2160 fmt:p010le sar:1/1 flags:0x0
> >      Last message repeated 3 times
> > [h264_qsv @ 0000019ee9995dc0] Using device qd1 (type qsv) with
> h264_qsv encoder.
> > [h264_qsv @ 0000019ee9995dc0] Encoder: input is system memory
> surface
> > [h264_qsv @ 0000019ee9995dc0] Using the average variable bitrate
> (AVBR) ratecontrol method
> > [h264_qsv @ 0000019ee9995dc0] Current pixel format is unsupported
> > [h264_qsv @ 0000019ee9995dc0] some encoding parameters are not
> supported by the QSV runtime. Please double check the input
> parameters.
> > Error initializing output stream 0:0 -- Error while opening encoder
> for output stream #0:0 - maybe incorrect parameters such as bit_rate,
> rate, width or height
> > [AVIOContext @ 0000019ef62b4000] Statistics: 0 bytes written, 0
> seeks, 0 writeouts
> > [AVIOContext @ 0000019ee995abc0] Statistics: 131146 bytes read, 2
> seeks
> > Conversion failed!
> >
> > softworkz
> 
> Alright, thanks for testing it. The commit message should mention the
> pixel format is being removed as it's unsupported, then.


The QuickSync architecture does not have any scaling/color conversion
capabilities at the encoder stage. There's the VPP filtering and they
also have a decoder-size fixed-function scaling which is separate from
VPP scaling (similar to CUVID). It's not yet exposed via ffmpeg, though.

Generally - when other video processing is required, it doesn't make 
much sense to do encoder-side scaling/conversion. You'll rather
want to scale down first and then do the other processing on the 
smaller frames.

The only case where I've seen an encoder-side scaling feature is 
MediaCodec, but well - there's no custom processing anyway.

Best,
softworkz