[FFmpeg-devel] [PATCH 2/2] avcodec/svq3: Reject dx/dy beyond 16bit

[Michael Niedermayer]

On Thu, Apr 27, 2017 at 03:22:53PM +0200, Nicolas George wrote:
> L'octidi 8 floréal, an CCXXV, Michael Niedermayer a écrit :
> > The code does use 16bit sized arrays later so larger deltas would not work
> > 
> > Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
> > ---
> >  libavcodec/svq3.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/libavcodec/svq3.c b/libavcodec/svq3.c
> > index 3e35fd73d6..76a465b9c0 100644
> > --- a/libavcodec/svq3.c
> > +++ b/libavcodec/svq3.c
> > @@ -551,7 +551,7 @@ static inline int svq3_mc_dir(SVQ3Context *s, int size, int mode,
> >                  dy = get_interleaved_se_golomb(&s->gb_slice);
> >                  dx = get_interleaved_se_golomb(&s->gb_slice);
> >  
> > -                if (dx == INVALID_VLC || dy == INVALID_VLC) {
> 
> > +                if (dx != (int16_t)dx || dy != (int16_t)dy) {
> 
> Is this cast not an undefined behaviour if dx is not in the range?

It is implementation defined (6.3.1.3.3 ISO/IEC 9899:TC3)

We require Twos-complement style definition for signed integers


> 
> An explicit "& 0x7FFF" may be better. Or using uint16_t, I do not know
> the code.

The value is signed, so these will not work without additional code

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Let us carefully observe those good qualities wherein our enemies excel us
and endeavor to excel them, by avoiding what is faulty, and imitating what
is excellent in them. -- Plutarch
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20170427/9e001efc/attachment.sig>