[FFmpeg-devel] [PATCH 4/5] avcodec/proresenc_anatoliy: do not write into alpha reserved bitfields

[Clément Bœsch]

On Mon, Jan 08, 2024 at 09:23:05PM +0100, Stefano Sabatini wrote:
> On date Sunday 2024-01-07 19:16:46 +0100, Clément Bœsch wrote:
> > This byte represents 4 reserved bits followed by 4 alpha_channel_type bits.
> > 
> > alpha_channel_type currently has 3 differents defined values: 0 (no
> > alpha), 1 (8b alpha), and 2 (16b alpha), all the other values are
> > reserved. The 4 initial reserved bits are expected to be 0.
> > ---
> >  libavcodec/proresenc_anatoliy.c             | 10 +---------
> >  tests/ref/vsynth/vsynth1-prores             |  2 +-
> >  tests/ref/vsynth/vsynth1-prores_444         |  2 +-
> >  tests/ref/vsynth/vsynth1-prores_444_int     |  2 +-
> >  tests/ref/vsynth/vsynth1-prores_int         |  2 +-
> >  tests/ref/vsynth/vsynth2-prores             |  2 +-
> >  tests/ref/vsynth/vsynth2-prores_444         |  2 +-
> >  tests/ref/vsynth/vsynth2-prores_444_int     |  2 +-
> >  tests/ref/vsynth/vsynth2-prores_int         |  2 +-
> >  tests/ref/vsynth/vsynth3-prores             |  2 +-
> >  tests/ref/vsynth/vsynth3-prores_444         |  2 +-
> >  tests/ref/vsynth/vsynth3-prores_444_int     |  2 +-
> >  tests/ref/vsynth/vsynth3-prores_int         |  2 +-
> >  tests/ref/vsynth/vsynth_lena-prores         |  2 +-
> >  tests/ref/vsynth/vsynth_lena-prores_444     |  2 +-
> >  tests/ref/vsynth/vsynth_lena-prores_444_int |  2 +-
> >  tests/ref/vsynth/vsynth_lena-prores_int     |  2 +-
> >  17 files changed, 17 insertions(+), 25 deletions(-)
> > 
> > diff --git a/libavcodec/proresenc_anatoliy.c b/libavcodec/proresenc_anatoliy.c
> > index 2223721aa0..1112cb26f1 100644
> > --- a/libavcodec/proresenc_anatoliy.c
> > +++ b/libavcodec/proresenc_anatoliy.c
> > @@ -794,15 +794,7 @@ static int prores_encode_frame(AVCodecContext *avctx, AVPacket *pkt,
> >                                        pict->color_trc, valid_trc, 0);
> >      *buf++ = int_from_list_or_default(avctx, "frame colorspace",
> >                                        pict->colorspace, valid_colorspace, 0);
> > -    if (avctx->profile >= AV_PROFILE_PRORES_4444) {
> > -        if (avctx->pix_fmt == AV_PIX_FMT_YUV444P10) {
> > -            *buf++ = 0xA0;/* src b64a and no alpha */
> > -        } else {
> > -            *buf++ = 0xA2;/* src b64a and 16b alpha */
> > -        }
> > -    } else {
> > -        *buf++ = 32;/* src v210 and no alpha */
> > -    }
> 
> > +    *buf++ = ctx->need_alpha ? 0x2 /* 16-bit alpha */ : 0;
> 
> Is 8-bit alpha mode supported? (it doesn't seem from a quic look).
> Should be good if not, thanks.

Yeah it's unsupported; you can look into put_alpha_diff() where abits is
hardcoded to 16 (ks encoder supports both though)

-- 
Clément B.