[FFmpeg-devel] [PATCH] avutil: Add NV61 pixel format and enable IO support for NV16 and NV61 pixel formats

Alexander Strasser eclipse7 at gmx.net
Fri Jul 10 14:43:30 EEST 2020


On 2020-07-07 23:09 -0400, Cristian Bicheru wrote:
> It's been some time since there was any activity on this thread so I'm
> wondering if this new patch with just the NV16 format is okay?

Only to be clear, I think adding NV61 too, would be more logical,
but in no way should my opinion be a blocker for this version
only adding NV16.

So if my statement wasn't wrongly considered a blocker, I guess
there is a lack of someone to review, test and push this patch.

I can't promise to test it soon and also think it's better if
someone with more experience of extending swscale would go ahead.

BTW isn't the whole libavutil part removed now?
If so commit message should be adapted and lavu version bump should
be removed.

Maybe bumping lsws minor is appropriate, I'm not exactly sure we
handled this consistently in the past.


Best regards,
  Alexander

>
> If there are any other suggestions I'd be glad to implement them.
>
> On Sat, Jun 20, 2020 at 1:59 PM Cristian Bicheru <c.bicheru0 at gmail.com> wrote:
> >
> > NV16 is an interleaved 4:2:2 8-bit format. There was some internal
> > NV16 support prior to this patch but the pixel format was not publicly
> > accessible.
> > ---
> >  libavutil/version.h                      |  2 +-
> >  libswscale/input.c                       |  1 +
> >  libswscale/output.c                      |  6 +++--
> >  libswscale/swscale_unscaled.c            | 43 ++++++++++++++++++++++++++++++++
> >  libswscale/utils.c                       |  1 +
> >  tests/ref/fate/filter-pixdesc-nv16       |  1 +
> >  tests/ref/fate/filter-pixfmts-copy       |  1 +
> >  tests/ref/fate/filter-pixfmts-crop       |  1 +
> >  tests/ref/fate/filter-pixfmts-field      |  1 +
> >  tests/ref/fate/filter-pixfmts-fieldorder |  1 +
> >  tests/ref/fate/filter-pixfmts-hflip      |  1 +
> >  tests/ref/fate/filter-pixfmts-il         |  1 +
> >  tests/ref/fate/filter-pixfmts-null       |  1 +
> >  tests/ref/fate/filter-pixfmts-pad        |  1 +
> >  tests/ref/fate/filter-pixfmts-scale      |  1 +
> >  tests/ref/fate/filter-pixfmts-vflip      |  1 +
> >  16 files changed, 61 insertions(+), 3 deletions(-)
> >  create mode 100644 tests/ref/fate/filter-pixdesc-nv16
> >
> > diff --git a/libavutil/version.h b/libavutil/version.h
> > index 3ce9b18..a63f79f 100644
> > --- a/libavutil/version.h
> > +++ b/libavutil/version.h
> > @@ -79,7 +79,7 @@
> >   */
> >
> >  #define LIBAVUTIL_VERSION_MAJOR  56
> > -#define LIBAVUTIL_VERSION_MINOR  55
> > +#define LIBAVUTIL_VERSION_MINOR  56
> >  #define LIBAVUTIL_VERSION_MICRO 100
> >
> >  #define LIBAVUTIL_VERSION_INT   AV_VERSION_INT(LIBAVUTIL_VERSION_MAJOR, \
> > diff --git a/libswscale/input.c b/libswscale/input.c
> > index 0bd1aa7..7f1a848 100644
> > --- a/libswscale/input.c
> > +++ b/libswscale/input.c
> > @@ -1114,6 +1114,7 @@ av_cold void ff_sws_init_input_funcs(SwsContext *c)
> >          break;
> >      case AV_PIX_FMT_NV12:
> >      case AV_PIX_FMT_NV24:
> > +    case AV_PIX_FMT_NV16:
> >          c->chrToYV12 = nv12ToUV_c;
> >          break;
> >      case AV_PIX_FMT_NV21:
> > diff --git a/libswscale/output.c b/libswscale/output.c
> > index 4ef436e..dc277f6 100644
> > --- a/libswscale/output.c
> > +++ b/libswscale/output.c
> > @@ -411,7 +411,8 @@ static void yuv2nv12cX_c(enum AVPixelFormat
> > dstFormat, const uint8_t *chrDither,
> >      int i;
> >
> >      if (dstFormat == AV_PIX_FMT_NV12 ||
> > -        dstFormat == AV_PIX_FMT_NV24)
> > +        dstFormat == AV_PIX_FMT_NV24 ||
> > +        dstFormat == AV_PIX_FMT_NV16)
> >          for (i=0; i<chrDstW; i++) {
> >              int u = chrDither[i & 7] << 12;
> >              int v = chrDither[(i + 3) & 7] << 12;
> > @@ -2586,7 +2587,8 @@ av_cold void ff_sws_init_output_funcs(SwsContext *c,
> >          *yuv2plane1 = yuv2plane1_8_c;
> >          *yuv2planeX = yuv2planeX_8_c;
> >          if (dstFormat == AV_PIX_FMT_NV12 || dstFormat == AV_PIX_FMT_NV21 ||
> > -            dstFormat == AV_PIX_FMT_NV24 || dstFormat == AV_PIX_FMT_NV42)
> > +            dstFormat == AV_PIX_FMT_NV24 || dstFormat == AV_PIX_FMT_NV42 ||
> > +            dstFormat == AV_PIX_FMT_NV16)
> >              *yuv2nv12cX = yuv2nv12cX_c;
> >      }
> >
> > diff --git a/libswscale/swscale_unscaled.c b/libswscale/swscale_unscaled.c
> > index 5fb572b..e83e92a 100644
> > --- a/libswscale/swscale_unscaled.c
> > +++ b/libswscale/swscale_unscaled.c
> > @@ -180,6 +180,39 @@ static int nv12ToPlanarWrapper(SwsContext *c,
> > const uint8_t *src[],
> >      return srcSliceH;
> >  }
> >
> > +static int planarToNv16Wrapper(SwsContext *c, const uint8_t *src[],
> > +                               int srcStride[], int srcSliceY,
> > +                               int srcSliceH, uint8_t *dstParam[],
> > +                               int dstStride[])
> > +{
> > +    uint8_t *dst = dstParam[1] + dstStride[1] * srcSliceY;
> > +
> > +    copyPlane(src[0], srcStride[0], srcSliceY, srcSliceH, c->srcW,
> > +              dstParam[0], dstStride[0]);
> > +
> > +    interleaveBytes(src[1], src[2], dst, c->chrSrcW, srcSliceH,
> > +                    srcStride[1], srcStride[2], dstStride[1]);
> > +
> > +    return srcSliceH;
> > +}
> > +
> > +static int nv16ToPlanarWrapper(SwsContext *c, const uint8_t *src[],
> > +                               int srcStride[], int srcSliceY,
> > +                               int srcSliceH, uint8_t *dstParam[],
> > +                               int dstStride[])
> > +{
> > +    uint8_t *dst1 = dstParam[1] + dstStride[1] * srcSliceY;
> > +    uint8_t *dst2 = dstParam[2] + dstStride[2] * srcSliceY;
> > +
> > +    copyPlane(src[0], srcStride[0], srcSliceY, srcSliceH, c->srcW,
> > +              dstParam[0], dstStride[0]);
> > +
> > +    deinterleaveBytes(src[1], dst1, dst2, c->chrSrcW, srcSliceH,
> > +                      srcStride[1], dstStride[1], dstStride[2]);
> > +
> > +    return srcSliceH;
> > +}
> > +
> >  static int planarToNv24Wrapper(SwsContext *c, const uint8_t *src[],
> >                                 int srcStride[], int srcSliceY,
> >                                 int srcSliceH, uint8_t *dstParam[],
> > @@ -1941,6 +1974,11 @@ void ff_get_unscaled_swscale(SwsContext *c)
> >          (dstFormat == AV_PIX_FMT_NV12 || dstFormat == AV_PIX_FMT_NV21)) {
> >          c->swscale = planarToNv12Wrapper;
> >      }
> > +    /* yv16_to_nv16 */
> > +    if ((srcFormat == AV_PIX_FMT_YUV422P || srcFormat ==
> > AV_PIX_FMT_YUVA422P) &&
> > +        dstFormat == AV_PIX_FMT_NV16) {
> > +        c->swscale = planarToNv16Wrapper;
> > +    }
> >      /* yv24_to_nv24 */
> >      if ((srcFormat == AV_PIX_FMT_YUV444P || srcFormat ==
> > AV_PIX_FMT_YUVA444P) &&
> >          (dstFormat == AV_PIX_FMT_NV24 || dstFormat == AV_PIX_FMT_NV42)) {
> > @@ -1951,6 +1989,11 @@ void ff_get_unscaled_swscale(SwsContext *c)
> >          (srcFormat == AV_PIX_FMT_NV12 || srcFormat == AV_PIX_FMT_NV21)) {
> >          c->swscale = nv12ToPlanarWrapper;
> >      }
> > +    /* nv16_to_yv16 */
> > +    if (dstFormat == AV_PIX_FMT_YUV422P &&
> > +        srcFormat == AV_PIX_FMT_NV16) {
> > +        c->swscale = nv16ToPlanarWrapper;
> > +    }
> >      /* nv24_to_yv24 */
> >      if (dstFormat == AV_PIX_FMT_YUV444P &&
> >          (srcFormat == AV_PIX_FMT_NV24 || srcFormat == AV_PIX_FMT_NV42)) {
> > diff --git a/libswscale/utils.c b/libswscale/utils.c
> > index dcd1dba..396a2aa 100644
> > --- a/libswscale/utils.c
> > +++ b/libswscale/utils.c
> > @@ -272,6 +272,7 @@ static const FormatEntry format_entries[] = {
> >      [AV_PIX_FMT_NV42]        = { 1, 1 },
> >      [AV_PIX_FMT_Y210LE]      = { 1, 0 },
> >      [AV_PIX_FMT_X2RGB10LE]   = { 1, 1 },
> > +    [AV_PIX_FMT_NV16]        = { 1, 1 },
> >  };
> >
> >  int sws_isSupportedInput(enum AVPixelFormat pix_fmt)
> > diff --git a/tests/ref/fate/filter-pixdesc-nv16
> > b/tests/ref/fate/filter-pixdesc-nv16
> > new file mode 100644
> > index 0000000..4c4587c
> > --- /dev/null
> > +++ b/tests/ref/fate/filter-pixdesc-nv16
> > @@ -0,0 +1 @@
> > +pixdesc-nv16        cbbc5d904b8eebad3705a325c3ec5734
> > diff --git a/tests/ref/fate/filter-pixfmts-copy
> > b/tests/ref/fate/filter-pixfmts-copy
> > index 1d7657c..1980137 100644
> > --- a/tests/ref/fate/filter-pixfmts-copy
> > +++ b/tests/ref/fate/filter-pixfmts-copy
> > @@ -56,6 +56,7 @@ grayf32le           fb6ea85bfbc8cd21c51fc0e110197294
> >  monob               8b04f859fee6a0be856be184acd7a0b5
> >  monow               54d16d2c01abfd72ecdb5e51e283937c
> >  nv12                8e24feb2c544dc26a20047a71e4c27aa
> > +nv16                22b1916c0694c4e2979bab8eb71f3d6b
> >  nv21                335d85c9af6110f26ae9e187a82ed2cf
> >  nv24                f30fc8d0ac40af69e119ea919a314572
> >  nv42                29a212f70f8780fe0eb99abcae81894d
> > diff --git a/tests/ref/fate/filter-pixfmts-crop
> > b/tests/ref/fate/filter-pixfmts-crop
> > index 8fc7614..edcb758 100644
> > --- a/tests/ref/fate/filter-pixfmts-crop
> > +++ b/tests/ref/fate/filter-pixfmts-crop
> > @@ -54,6 +54,7 @@ gray9le             4d1932d4968a248584f5e39c25f1dd43
> >  grayf32be           cf40ec06a8abe54852b7f85a00549eec
> >  grayf32le           b672526c9da9c8959ab881f242f6890a
> >  nv12                92cda427f794374731ec0321ee00caac
> > +nv16                3264b16aaae554c21f052102b491c13b
> >  nv21                1bcfc197f4fb95de85ba58182d8d2f69
> >  nv24                514c8f12082f0737e558778cbe7de258
> >  nv42                ece9baae1c5de579dac2c66a89e08ef3
> > diff --git a/tests/ref/fate/filter-pixfmts-field
> > b/tests/ref/fate/filter-pixfmts-field
> > index ce8e535..46462b0 100644
> > --- a/tests/ref/fate/filter-pixfmts-field
> > +++ b/tests/ref/fate/filter-pixfmts-field
> > @@ -56,6 +56,7 @@ grayf32le           6b856bdbf2a2bfcd2bc7d50f109daaf0
> >  monob               2129cc72a484d7e10a44de9117aa9f80
> >  monow               03d783611d265cae78293f88ea126ea1
> >  nv12                16f7a46708ef25ebd0b72e47920cc11e
> > +nv16                34f36b03f5fccf4eac147b26bbc0a5e5
> >  nv21                7294574037cc7f9373ef5695d8ebe809
> >  nv24                3b100fb527b64ee2b2d7120da573faf5
> >  nv42                1841ce853152d86b27c130f319ea0db2
> > diff --git a/tests/ref/fate/filter-pixfmts-fieldorder
> > b/tests/ref/fate/filter-pixfmts-fieldorder
> > index 90d36ad..55a0043 100644
> > --- a/tests/ref/fate/filter-pixfmts-fieldorder
> > +++ b/tests/ref/fate/filter-pixfmts-fieldorder
> > @@ -53,6 +53,7 @@ gray9be             ec877f5bcf0ea275a6f36c12cc9adf11
> >  gray9le             fba944fde7923d5089f4f52d12988b9e
> >  grayf32be           1aa7960131f880c54fe3c77f13448674
> >  grayf32le           4029ac9d197f255794c1b9e416520fc7
> > +nv16                085deb984ab986eb5cc961fe265e30c0
> >  nv24                4fdbef26042c77f012df114e666efdb2
> >  nv42                59608290fece913e6b7d61edf581a529
> >  rgb0                2e3d8c91c7a83d451593dfd06607ff39
> > diff --git a/tests/ref/fate/filter-pixfmts-hflip
> > b/tests/ref/fate/filter-pixfmts-hflip
> > index 0d40b93..584ed39 100644
> > --- a/tests/ref/fate/filter-pixfmts-hflip
> > +++ b/tests/ref/fate/filter-pixfmts-hflip
> > @@ -54,6 +54,7 @@ gray9le             424fc581947bc8c357c9ec5e3c1c04d1
> >  grayf32be           a69add7bbf892a71fe81b3b75982dbe2
> >  grayf32le           4563e176a35dc8a8a07e0829fad5eb88
> >  nv12                801e58f1be5fd0b5bc4bf007c604b0b4
> > +nv16                06ba714cb8b220c203f5898ef39abf93
> >  nv21                9f10dfff8963dc327d3395af21f0554f
> >  nv24                f0c5b2f42970f8d4003621d8857a872f
> >  nv42                4dcf9aec82b110712b396a8b365dcb13
> > diff --git a/tests/ref/fate/filter-pixfmts-il b/tests/ref/fate/filter-pixfmts-il
> > index d1bc866..96c9409 100644
> > --- a/tests/ref/fate/filter-pixfmts-il
> > +++ b/tests/ref/fate/filter-pixfmts-il
> > @@ -56,6 +56,7 @@ grayf32le           8bf3d295c3ffd53da0e06d0702e7c1ca
> >  monob               faba75df28033ba7ce3d82ff2a99ee68
> >  monow               6e9cfb8d3a344c5f0c3e1d5e1297e580
> >  nv12                3c3ba9b1b4c4dfff09c26f71b51dd146
> > +nv16                355d055f91793a171302021b3fc486b0
> >  nv21                ab586d8781246b5a32d8760a61db9797
> >  nv24                554153c71d142e3fd8e40b7dcaaec229
> >  nv42                d699724c8deaeb4f87faf2766512eec3
> > diff --git a/tests/ref/fate/filter-pixfmts-null
> > b/tests/ref/fate/filter-pixfmts-null
> > index 1d7657c..1980137 100644
> > --- a/tests/ref/fate/filter-pixfmts-null
> > +++ b/tests/ref/fate/filter-pixfmts-null
> > @@ -56,6 +56,7 @@ grayf32le           fb6ea85bfbc8cd21c51fc0e110197294
> >  monob               8b04f859fee6a0be856be184acd7a0b5
> >  monow               54d16d2c01abfd72ecdb5e51e283937c
> >  nv12                8e24feb2c544dc26a20047a71e4c27aa
> > +nv16                22b1916c0694c4e2979bab8eb71f3d6b
> >  nv21                335d85c9af6110f26ae9e187a82ed2cf
> >  nv24                f30fc8d0ac40af69e119ea919a314572
> >  nv42                29a212f70f8780fe0eb99abcae81894d
> > diff --git a/tests/ref/fate/filter-pixfmts-pad
> > b/tests/ref/fate/filter-pixfmts-pad
> > index 9a5db82..d536683 100644
> > --- a/tests/ref/fate/filter-pixfmts-pad
> > +++ b/tests/ref/fate/filter-pixfmts-pad
> > @@ -22,6 +22,7 @@ gray14le            af3f2f911c71cb34a8179a3291b5c90f
> >  gray16le            468bda6155bdc7a7a20c34d6e599fd16
> >  gray9le             f8f3dfe31ca5fcba828285bceefdab9a
> >  nv12                381574979cb04be10c9168540310afad
> > +nv16                d3a50501d2ea8535489fd5ec49e7866d
> >  nv21                0fdeb2cdd56cf5a7147dc273456fa217
> >  nv24                193b9eadcc06ad5081609f76249b3e47
> >  nv42                1738ad3c31c6c16e17679f5b09ce4677
> > diff --git a/tests/ref/fate/filter-pixfmts-scale
> > b/tests/ref/fate/filter-pixfmts-scale
> > index d7020ad..aed44a2 100644
> > --- a/tests/ref/fate/filter-pixfmts-scale
> > +++ b/tests/ref/fate/filter-pixfmts-scale
> > @@ -56,6 +56,7 @@ grayf32le           2ff1b84023e820307b1ba7a9550115bc
> >  monob               f01cb0b623357387827902d9d0963435
> >  monow               35c68b86c226d6990b2dcb573a05ff6b
> >  nv12                b118d24a3653fe66e5d9e079033aef79
> > +nv16                68e757396b62b84aad657274b8f6ce15
> >  nv21                c74bb1c10dbbdee8a1f682b194486c4d
> >  nv24                2aa6e805bf6d4179ed8d7dea37d75db3
> >  nv42                80714d1eb2d8bcaeab3abc3124df1abd
> > diff --git a/tests/ref/fate/filter-pixfmts-vflip
> > b/tests/ref/fate/filter-pixfmts-vflip
> > index 732db8d..7e75345 100644
> > --- a/tests/ref/fate/filter-pixfmts-vflip
> > +++ b/tests/ref/fate/filter-pixfmts-vflip
> > @@ -56,6 +56,7 @@ grayf32le           8e6c048a5b3b8b26d3a5ddfce255f3f6
> >  monob               7810c4857822ccfc844d78f5e803269a
> >  monow               90a947bfcd5f2261e83b577f48ec57b1
> >  nv12                261ebe585ae2aa4e70d39a10c1679294
> > +nv16                f20f3448c900847aaff74429196f5a00
> >  nv21                2909feacd27bebb080c8e0fa41795269
> >  nv24                334420b9d3df84499d2ca16bb66eed2b
> >  nv42                ba4063e2795c17fea3c8a646b01fd1f5
> > --
> > 2.7.4
> >
> > On Sat, Jun 20, 2020 at 2:44 AM Alexander Strasser <eclipse7 at gmx.net> wrote:
> > >
> > >
> > >
> > > Am 20. Juni 2020 00:23:53 MESZ schrieb Hendrik Leppkes <h.leppkes at gmail.com>:
> > > >On Fri, Jun 19, 2020 at 9:58 PM Alexander Strasser <eclipse7 at gmx.net>
> > > >wrote:
> > > >>
> > > >> How do others think about adding support for more pixel formats?
> > > >>
> > > >
> > > >A new pixel format should present a clear improvement, a use-case you
> > > >couldn't do before, or could only do with a performance penalty or
> > > >whatever.
> > >
> > > Thanks for your quick reply. Generally I agree here.
> > >
> > > >If everything that you can do with NV61 you could also do with NV16, a
> > > >format we already have, which value does adding it add to the project
> > >
> > > In this case it's true, that you can't do anything special with NV61 you can't do with NV16, but I think they can also be viewed as the same format.
> > >
> > > So if you add one, you also add the other. Like when I would add NV12 support I would also add NV21 support.
> > >
> > > It has the added value, that whenever a user needs one or the other as input or output it would be available.
> > >
> > > On the implementation side it is kind of easy to test, that both work the same with chroma interleaving swapped.
> > >
> > > But maybe I misunderstand something important here?
> > >
> > > >Completionism should not be a goal. There are hundreds of obscure
> > > >pixel formats that we have no business all adding.
> > >
> > > I fully agree that completionism should not be a goal.
> > >
> > >
> > >   Alexander


More information about the ffmpeg-devel mailing list