[FFmpeg-devel] [PATCH]lavc/r210enc: Fix undefined behaviour encoding r10k

[Carl Eugen Hoyos]

Am Mo., 1. Juli 2019 um 20:27 Uhr schrieb Michael Niedermayer
<michael at niedermayer.cc>:
>
> On Mon, Jul 01, 2019 at 01:08:34PM +0200, Carl Eugen Hoyos wrote:
> > Hi!
> >
> > Attached patch fixes an invalid left shift reported in ticket #7982.
> >
> > Please comment, Carl Eugen
>
> >  r210enc.c |    6 +++---
> >  1 file changed, 3 insertions(+), 3 deletions(-)
> > 9db2195a8e6d3c90726c2bf92119ff6043d4dd99  0001-lavc-r210enc-Fix-undefined-behaviour-encoding-r10k.patch
> > From cfe1ae9a1e95dee72ff0e0a45ce8caab98a1bd0b Mon Sep 17 00:00:00 2001
> > From: Carl Eugen Hoyos <ceffmpeg at gmail.com>
> > Date: Mon, 1 Jul 2019 13:06:02 +0200
> > Subject: [PATCH] lavc/r210enc: Fix undefined behaviour encoding r10k.
> >
> > Fixes the following ubsan error:
> > libavcodec/r210enc.c:69:28: runtime error: left shift of 522 by 22 places cannot be represented in type 'int'
> >
> > Fixes ticket #7982.
> > ---
> >  libavcodec/r210enc.c | 6 +++---
> >  1 file changed, 3 insertions(+), 3 deletions(-)
> >
> > diff --git a/libavcodec/r210enc.c b/libavcodec/r210enc.c
> > index 02412f3684..95049089bc 100644
> > --- a/libavcodec/r210enc.c
> > +++ b/libavcodec/r210enc.c
> > @@ -60,9 +60,9 @@ static int encode_frame(AVCodecContext *avctx, AVPacket *pkt,
> >          uint16_t *srcb = (uint16_t *)srcb_line;
> >          for (j = 0; j < avctx->width; j++) {
> >              uint32_t pixel;
> > -            uint16_t r = *srcr++;
> > -            uint16_t g = *srcg++;
> > -            uint16_t b = *srcb++;
> > +            uint32_t r = *srcr++;
> > +            uint32_t g = *srcg++;
> > +            uint32_t b = *srcb++;
> >              if (avctx->codec_id == AV_CODEC_ID_R210)
> >                  pixel = (r << 20) | (g << 10) | b;
>
> I think plain unsigned is a less confusing choice here, but either is
> ok

Pushed with unsigned.

Thank you, Carl Eugen