[FFmpeg-devel] [PATCH v4 1/2] libavcodec/libaomenc.c: Support gray input

[James Zern]

Hi,

On Tue, Apr 7, 2020 at 5:14 PM Ryo Hirafuji <ryo.hirafuji at gmail.com> wrote:
>
> From: Ryo Hirafuji <psi at 7io.org>
>
> AV1 decoders, libaomdec and libdav1d, both support grayscale image.
> However, libaomenc does not support it yet.
> In this patch, I add a grayscale image support also to libaomenc.
>
> Fixes ticket #7599
> ---
>  libavcodec/libaomenc.c | 40 +++++++++++++++++++++++++++++++++-------
>  1 file changed, 33 insertions(+), 7 deletions(-)
>

This should bump the micro version number in libavcodec/version.h.

> [...]
> @@ -979,12 +986,27 @@ static int aom_encode(AVCodecContext *avctx, AVPacket *pkt,
>
>      if (frame) {
>          rawimg                      = &ctx->rawimg;
> -        rawimg->planes[AOM_PLANE_Y] = frame->data[0];
> -        rawimg->planes[AOM_PLANE_U] = frame->data[1];
> -        rawimg->planes[AOM_PLANE_V] = frame->data[2];
> -        rawimg->stride[AOM_PLANE_Y] = frame->linesize[0];
> -        rawimg->stride[AOM_PLANE_U] = frame->linesize[1];
> -        rawimg->stride[AOM_PLANE_V] = frame->linesize[2];
> +        if (frame->format == AV_PIX_FMT_GRAY8 ||
> +            frame->format == AV_PIX_FMT_GRAY10 ||
> +            frame->format == AV_PIX_FMT_GRAY12) {
> +            rawimg->monochrome = 1;
> +            // Information of U and V planes are ignored,
> +            // but must point some valid pointer to avoid SIGSEGV of libaom.

That's annoying. I filed a bug to track [1]. The monochrome flag
itself seems unnecessary for the library rather than just an image
format, but that's another discussion.
This reads a little strangely to me, maybe something like: U and V
information is ignored, but must point to valid buffers...?

[1] https://crbug.com/aomedia/2639