[MPlayer-dev-eng] [PATCH] segfault in vf_scale

[Reimar Döffinger]

Hi,
> > > > especially in combination with -vf pp, vf_scale can segfault because the
> > > > mpis can have different height, especially when height is not divisible
> > > > by 16.
> > > 
> > > huh? imo this shouldn't happen... if it does i'd consider it a bug, or
> > > more like a bug in the whole vf layer, which is very possible..
> > 
> > Not sure if you understood me correctly, I wasn't very precise. From
> > what I can see, the decoder can ask for an mpi that is larger the actual
> > image size. But when vf_scale does no scaling, it allocates a output mpi
> > of the real video size - wich will be too small when it tries to convert
> > mpi->h lines.
> > A solution could be to save the height from the config call and use that
> > instead...
> 
> passing an image with mpi->h different from the height requested in
> config is ILLEGAL!!! this needs to be fixed.
> 
> > Also the question is, which value should mpi->h contain? And which one
> > mpi->height? They currently see to be used for the same things, which
> > makes <0 sense to me :-(
> 
> right, this is broken. mpi->height can be larger in theory, but that's
> a broken design too. the only correct way i can see when a codec or
> filter needs extra junk space at the edges is to specifically request
> that when requesting the image, but NOT have it show up in the image
> width/height at all, just the actual arrangement in memory. this is
> the way my g2 filter layer was being designed but it's stalled
> indefinitely.
> 
> in the mean time, fix the broken code that's requesting different size
> from what it config()ured... i see 2 possible ways:
> 
> 1. skip postprocessing of partial blocks on stupidly-missized movies.
>    this is what i like.
> 
> 2. have vf_pp allocate its own buffer with extra working room when
>    sizes are idiotic, and then use EXPORT type image.
> 
> maybe there's another way to do it too. iirc the libpostproc code is
> very flexible about boundary conditions.

Actually the problem is not in the postprocessing code, it shows up when
using postprocessing because then the decoder buffer isn't used directly
anymore but instead the video is copied into the vo's (then too small)
mpi.
The attached patch seems to fix things for me. And don't dare to tell me
that's the wrong way to fix without providing a better patch!

Greetings,
Reimar Döffinger
-------------- next part --------------
Index: libmpcodecs/vd_ffmpeg.c
===================================================================
RCS file: /cvsroot/mplayer/main/libmpcodecs/vd_ffmpeg.c,v
retrieving revision 1.136
diff -u -r1.136 vd_ffmpeg.c
--- libmpcodecs/vd_ffmpeg.c	11 Oct 2004 16:23:20 -0000	1.136
+++ libmpcodecs/vd_ffmpeg.c	29 Oct 2004 11:10:44 -0000
@@ -578,6 +578,8 @@
 
     mpi= mpcodecs_get_image(sh,type, flags,
 			(width+align)&(~align), (height+align)&(~align));
+    mpi->w = width;
+    mpi->h = height;
 
     // ok, let's see what did we get:
     if(  mpi->flags&MP_IMGFLAG_DRAW_CALLBACK &&