[FFmpeg-devel] [PATCH] swscale: Check whether slice height is multiple of chroma subsample.

[Ramiro Polla]

Hi,

Check whether slice height is multiple of chroma subsample.

Currently if we take a 410p image and rescale with sliceH = 1, the
output is black&white. I did this using the test tool at [1] (renamed
to swscale-test2) and the command:
./swscale-test2 -i input.bmp -sws_flags bicubic -pix_fmt yuv410p -s
320x320 -scale -sliceH 1 -pix_fmt bgr24 -scale -o output.bmp

I found it weird that this code in utils.c expects the slices to start
on subsample boundaries (by shifting nextSlice back and forth), but
sws_scale() doesn't:
    // calculate buffer sizes so that they won't run out while
handling these damn slices
    c->vLumBufSize= c->vLumFilterSize;
    c->vChrBufSize= c->vChrFilterSize;
    for (i=0; i<dstH; i++) {
        int chrI= i*c->chrDstH / dstH;
        int nextSlice= FFMAX(c->vLumFilterPos[i   ] + c->vLumFilterSize - 1,
                           ((c->vChrFilterPos[chrI] +
c->vChrFilterSize - 1)<<c->chrSrcVSubSample));

        nextSlice>>= c->chrSrcVSubSample;
        nextSlice<<= c->chrSrcVSubSample;
        if (c->vLumFilterPos[i   ] + c->vLumBufSize < nextSlice)
            c->vLumBufSize= nextSlice - c->vLumFilterPos[i];
        if (c->vChrFilterPos[chrI] + c->vChrBufSize <
(nextSlice>>c->chrSrcVSubSample))
            c->vChrBufSize= (nextSlice>>c->chrSrcVSubSample) -
c->vChrFilterPos[chrI];
    }

Ramiro Polla
[1] http://article.gmane.org/gmane.comp.video.ffmpeg.devel/102702
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0004-Check-whether-slice-height-is-multiple-of-chroma-sub.patch
Type: text/x-diff
Size: 938 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20100223/4fe001a4/attachment.patch>