[FFmpeg-devel] [PATCH v2 1/2] lavu/hwcontext_qsv: add support for AV_PIX_FMT_VUYX on Linux

[Xiang, Haihao]

From: Haihao Xiang <haihao.xiang at intel.com>

AV_PIX_FMT_VUYX is used for 8bit 4:4:4 content in FFmpeg VAAPI, so
AV_PIX_FMT_VUYX should be used for 8bit 4:4:4 content in FFmpeg QSV too
because QSV is based on VAAPI on Linux. However the SDK only declares
support for AYUV and does nothing with the alpha, so this commit fudged
a mapping between AV_PIX_FMT_VUYX and MFX_FOURCC_AYUV.
---
 libavutil/hwcontext_qsv.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/libavutil/hwcontext_qsv.c b/libavutil/hwcontext_qsv.c
index 510f422562..9fa0dfa1c0 100644
--- a/libavutil/hwcontext_qsv.c
+++ b/libavutil/hwcontext_qsv.c
@@ -119,6 +119,10 @@ static const struct {
                        MFX_FOURCC_YUY2 },
     { AV_PIX_FMT_Y210,
                        MFX_FOURCC_Y210 },
+    // VUYX is used for VAAPI child device,
+    // the SDK only delares support for AYUV
+    { AV_PIX_FMT_VUYX,
+                       MFX_FOURCC_AYUV },
 #endif
 };
 
@@ -1502,6 +1506,14 @@ static int map_frame_to_surface(const AVFrame *frame, mfxFrameSurface1 *surface)
         surface->Data.U16 = (mfxU16 *)frame->data[0] + 1;
         surface->Data.V16 = (mfxU16 *)frame->data[0] + 3;
         break;
+    case AV_PIX_FMT_VUYX:
+        surface->Data.V = frame->data[0];
+        surface->Data.U = frame->data[0] + 1;
+        surface->Data.Y = frame->data[0] + 2;
+        // Only set Data.A to a valid address, the SDK doesn't
+        // use the value from the frame.
+        surface->Data.A = frame->data[0] + 3;
+        break;
 #endif
     default:
         return MFX_ERR_UNSUPPORTED;
-- 
2.17.1