[FFmpeg-devel] [PATCH 1/2] lavu/hwcontext_qsv: add support for AV_PIX_FMT_VUYX

[Xiang, Haihao]

On Mon, 2022-08-29 at 14:01 +0000, Xiang, Haihao wrote:
> On Mon, 2022-08-29 at 08:17 -0300, James Almer wrote:
> > On 8/29/2022 4:27 AM, Xiang, Haihao wrote:
> > > From: Haihao Xiang <haihao.xiang at intel.com>
> > > 
> > > AV_PIX_FMT_VUYX is used in FFmpeg for 8bit 4:4:4 content on Intel HW,
> > > and MFX_FOURCC_AYUV is used in the SDK
> > 
> > Sounds like you want the VUYA pixfmt instead.
> 
> AV_PIX_FMT_VUYX is used for 4:4:4 content in the VAAPI path, AV_PIX_FMT_VUYX
> is
> expected when creating vaapi urface. QSV is based on 
> VAAPI under Linux, so AV_PIX_FMT_VUYX is expected too in the QSV path when
> creating vaapi surface.
> 
> > 
> > > ---
> > >   libavutil/hwcontext_qsv.c | 8 ++++++++
> > >   1 file changed, 8 insertions(+)
> > > 
> > > diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c
> > > index 510f422562..a3350eae0f 100644
> > > --- a/libavutil/hwcontext_qsv.c
> > > +++ b/libavutil/hwcontext_qsv.c
> > > @@ -119,6 +119,8 @@ static const struct {
> > >                          MFX_FOURCC_YUY2 },
> > >       { AV_PIX_FMT_Y210,
> > >                          MFX_FOURCC_Y210 },
> > > +    { AV_PIX_FMT_VUYX,
> > > +                       MFX_FOURCC_AYUV },
> > >   #endif
> > >   };
> > >   
> > > @@ -1502,6 +1504,12 @@ static int map_frame_to_surface(const AVFrame
> > > *frame,
> > > mfxFrameSurface1 *surface)
> > >           surface->Data.U16 = (mfxU16 *)frame->data[0] + 1;
> > >           surface->Data.V16 = (mfxU16 *)frame->data[0] + 3;
> > >           break;
> > > +    case AV_PIX_FMT_VUYX:
> > > +        surface->Data.V = frame->data[0];
> > > +        surface->Data.U = frame->data[0] + 1;
> > > +        surface->Data.Y = frame->data[0] + 2;
> > > +        surface->Data.A = frame->data[0] + 3;
> > 
> > This will go wrong with VUYX. You need to use AV_PIX_FMT_VUYA.
> 
> frame->data[0] + 3 is valid even if alpha channel is ignored in VUYX. Intel HW
> doesn't use the data in alpha channel actually, but the SDK uses Microsoft
> pixel
> format AYUV which is the alpha version, here set a valid address to alpha
> channel when mapping a VUYX AVFrame to a AYUV mfx surface. 
> 

Is there more comment about this patch ? 

Thanks
Haihao