[MPlayer-dev-eng] [PATCH] segfault in vf_scale

[D Richard Felker III]

On Thu, Oct 21, 2004 at 10:47:22PM +0200, Reimar Döffinger wrote:
> Hi,
> 
> > > especially in combination with -vf pp, vf_scale can segfault because the
> > > mpis can have different height, especially when height is not divisible
> > > by 16.
> > 
> > huh? imo this shouldn't happen... if it does i'd consider it a bug, or
> > more like a bug in the whole vf layer, which is very possible..
> 
> Not sure if you understood me correctly, I wasn't very precise. From
> what I can see, the decoder can ask for an mpi that is larger the actual
> image size. But when vf_scale does no scaling, it allocates a output mpi
> of the real video size - wich will be too small when it tries to convert
> mpi->h lines.
> A solution could be to save the height from the config call and use that
> instead...

passing an image with mpi->h different from the height requested in
config is ILLEGAL!!! this needs to be fixed.

> Also the question is, which value should mpi->h contain? And which one
> mpi->height? They currently see to be used for the same things, which
> makes <0 sense to me :-(

right, this is broken. mpi->height can be larger in theory, but that's
a broken design too. the only correct way i can see when a codec or
filter needs extra junk space at the edges is to specifically request
that when requesting the image, but NOT have it show up in the image
width/height at all, just the actual arrangement in memory. this is
the way my g2 filter layer was being designed but it's stalled
indefinitely.

in the mean time, fix the broken code that's requesting different size
from what it config()ured... i see 2 possible ways:

1. skip postprocessing of partial blocks on stupidly-missized movies.
   this is what i like.

2. have vf_pp allocate its own buffer with extra working room when
   sizes are idiotic, and then use EXPORT type image.

maybe there's another way to do it too. iirc the libpostproc code is
very flexible about boundary conditions.

rich