[FFmpeg-devel] [PATCH 1/3] swscale/input: Fix several invalid shifts related to rgb2yuv constants
Michael Niedermayer
michael at niedermayer.cc
Wed Jan 22 19:44:17 EET 2020
On Wed, Jan 22, 2020 at 03:42:25PM +0100, Andreas Rheinhardt wrote:
> On Tue, Jan 21, 2020 at 11:18 PM Michael Niedermayer <michael at niedermayer.cc>
> wrote:
>
> > Fixes: Invalid shifts
> > Fixes: #8140
> > Fixes: #8146
> >
> > Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
> > ---
> > libswscale/input.c | 8 ++++----
> > 1 file changed, 4 insertions(+), 4 deletions(-)
> >
> > diff --git a/libswscale/input.c b/libswscale/input.c
> > index 064f8da314..159f70307d 100644
> > --- a/libswscale/input.c
> > +++ b/libswscale/input.c
> > @@ -286,8 +286,8 @@ static av_always_inline void
> > rgb16_32ToUV_c_template(int16_t *dstU,
> > int gsh, int bsh,
> > int S,
> > int32_t *rgb2yuv)
> > {
> > - const int ru = rgb2yuv[RU_IDX] << rsh, gu = rgb2yuv[GU_IDX] <<
> > gsh, bu = rgb2yuv[BU_IDX] << bsh,
> > - rv = rgb2yuv[RV_IDX] << rsh, gv = rgb2yuv[GV_IDX] <<
> > gsh, bv = rgb2yuv[BV_IDX] << bsh;
> > + const int ru = rgb2yuv[RU_IDX] * (1 << rsh), gu =
> > rgb2yuv[GU_IDX] * (1 << gsh), bu = rgb2yuv[BU_IDX] * (1 << bsh),
> > + rv = rgb2yuv[RV_IDX] * (1 << rsh), gv =
> > rgb2yuv[GV_IDX] * (1 << gsh), bv = rgb2yuv[BV_IDX] * (1 << bsh);
> > const unsigned rnd = (256u<<((S)-1)) + (1<<(S-7));
> > int i;
> >
> > @@ -314,8 +314,8 @@ static av_always_inline void
> > rgb16_32ToUV_half_c_template(int16_t *dstU,
> > int gsh, int
> > bsh, int S,
> > int32_t
> > *rgb2yuv)
> > {
> > - const int ru = rgb2yuv[RU_IDX] << rsh, gu = rgb2yuv[GU_IDX] <<
> > gsh, bu = rgb2yuv[BU_IDX] << bsh,
> > - rv = rgb2yuv[RV_IDX] << rsh, gv = rgb2yuv[GV_IDX] <<
> > gsh, bv = rgb2yuv[BV_IDX] << bsh,
> > + const int ru = rgb2yuv[RU_IDX] * (1 << rsh), gu =
> > rgb2yuv[GU_IDX] * (1 << gsh), bu = rgb2yuv[BU_IDX] * (1 << bsh),
> > + rv = rgb2yuv[RV_IDX] * (1 << rsh), gv =
> > rgb2yuv[GV_IDX] * (1 << gsh), bv = rgb2yuv[BV_IDX] * (1 << bsh),
> > maskgx = ~(maskr | maskb);
> > const unsigned rnd = (256U<<(S)) + (1<<(S-6));
> > int i;
> > --
>
> When I looked into this last year, I was unable to ascertain that the
> multiplication wouldn't overflow (in which case one would need to go the
> unsigned route, most easily by making the rgb2yuv parameter a uint32_t *).
> Can you?
If there is a situation where these overflow then this needs to be looked
at with a testcase. Simply using unsigned would likely not produce a
outcome the user wants
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
The smallest minority on earth is the individual. Those who deny
individual rights cannot claim to be defenders of minorities. - Ayn Rand
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20200122/7d270008/attachment.sig>
More information about the ffmpeg-devel
mailing list