[FFmpeg-devel] libavfilter/vsrc_testsrc : fix hang with testsrc2 and small size

[Martin Vignali]

2017-10-28 18:49 GMT+02:00 Nicolas George <george at nsup.org>:

> Le septidi 7 brumaire, an CCXXVI, Martin Vignali a écrit :
> > Hello,
> >
> > Following another discussion,
> > the testsrc2 filter can hang with small size
> >
> > Possible fix in attach
> >
> > Martin
>
> > From 3eaa767833647ab8ce270e091925557543ae5d35 Mon Sep 17 00:00:00 2001
> > From: Martin Vignali <martin.vignali at gmail.com>
> > Date: Sat, 28 Oct 2017 18:33:26 +0200
> > Subject: [PATCH 2/3] libavfilter/vsrc_testsrc : fix hang with testsrc2
> and
> >  small size
> >
> > ---
> >  libavfilter/vsrc_testsrc.c | 7 ++++---
> >  1 file changed, 4 insertions(+), 3 deletions(-)
> >
> > diff --git a/libavfilter/vsrc_testsrc.c b/libavfilter/vsrc_testsrc.c
> > index fe0d50aa41..81e8ce2e69 100644
> > --- a/libavfilter/vsrc_testsrc.c
> > +++ b/libavfilter/vsrc_testsrc.c
> > @@ -851,9 +851,10 @@ static void test2_fill_picture(AVFilterContext
> *ctx, AVFrame *frame)
> >      {
> >          unsigned xmin = av_rescale(5, s->w, 8);
> >          unsigned xmax = av_rescale(7, s->w, 8);
>
> > -        unsigned ymin = av_rescale(5, s->h, 8);
> > -        unsigned ymax = av_rescale(7, s->h, 8);
> > -        unsigned x, y, i, r;
> > +        int ymin = av_rescale(5, s->h, 8);
> > +        int ymax = av_rescale(7, s->h, 8);
> > +        unsigned x, i, r;
> > +        int y;
>
> I cannot peruse the source code for now, but I do not think that making
> signed variables that are semantically unsigned can be a correct fix. At
> the very least the commit message would need to explain why it changes a
> thing.
>
>
Don't know if there is a better way to fix it
but if the variable stay unsigned
this for loop create the problem
for (y = ymin; y < ymax - 15; y += 16) {

Martin