[Ffmpeg-devel] [PATCH] division by zero during rv20 decoding

[Reimar Döffinger]

Hello,
On Fri, Mar 09, 2007 at 11:31:38PM +0100, Michael Niedermayer wrote:
> On Fri, Mar 09, 2007 at 09:54:42PM +0100, Reimar D?ffinger wrote:
> > On Fri, Mar 09, 2007 at 09:08:30PM +0100, Michael Niedermayer wrote:
> > > On Fri, Mar 09, 2007 at 08:56:24PM +0100, Reimar D?ffinger wrote:
> > > > currently rv10.c, line 458 sets a new dimension without
> > > > doing avcodec_check_dimensions on it first.
> > > > This can cause a division by zero when playing in MPlayer, see
> > > > http://bugzilla.mplayerhq.hu/show_bug.cgi?id=773
> > > > Attached patch (to be improved by the maintainer ;-) ) fixes it, though
> > > > none of those detected resolution should happen (and actually they only
> > > > happen with MPlayer - couldn't test with -demuxer lavf since that
> > > > crashes almost instantly, I might look into that if I am bored somewhen
> > > > later).
> > > > 
> > > > Greetings,
> > > > Reimar D?ffinger
> > > 
> > > > Index: libavcodec/rv10.c
> > > > ===================================================================
> > > > --- libavcodec/rv10.c	(revision 8306)
> > > > +++ libavcodec/rv10.c	(working copy)
> > > > @@ -452,7 +452,8 @@
> > > >              new_w= s->width; //FIXME wrong we of course must save the original in the context
> > > >              new_h= s->height;
> > > >          }
> > > > -        if(new_w != s->width || new_h != s->height){
> > > > +        if((new_w != s->width || new_h != s->height) &&
> > > > +           avcodec_check_dimensions(s->avctx, new_h, new_w) >= 0){
> > > >              av_log(s->avctx, AV_LOG_DEBUG, "attempting to change resolution to %dx%d\n", new_w, new_h);
> > > 
> > > id rather:
> > > 
> > > if(avcodec_check_dimensions(s->avctx, new_h, new_w) < 0)
> > >     return -1;
> > 
> > Anything that does not actually set the invalid dimension works for
> > avoiding the crash AFAICT (specifically tested your suggestion as in attached
> > patch), it's just cosmetic decisions like whether to place the check before
> > or after the debug message or print an error message and whatever, that's what
> > I mostly meant by "to be improved by the maintainer".
> 
> looks ok

Applied.

Greetings,
Reimar D?ffinger