[FFmpeg-devel] [PATCH]Fix linesize for pix_fmt pal8

[Michael Niedermayer]

On Tue, Jul 03, 2012 at 05:57:45PM +0200, Michael Niedermayer wrote:
> On Sat, Jun 30, 2012 at 11:41:39PM +0200, Stefano Sabatini wrote:
> > On date Wednesday 2012-06-20 07:33:04 +0000, Carl Eugen Hoyos encoded:
> > > Stefano Sabatini <stefasab <at> gmail.com> writes:
> > > 
> > > > Check attached patch, it should fix the problem when coupled 
> > > > with the new imgutils functions.
> > > 
> > > The patch breaks decoding some of the samples from ticket #823.
> > 
> > New try (I still don't like very much this approach), but at least I
> > realized that changing the pal8 output alignment is just wrong. And we
> > could use a better heuristic for detecting the linesize (for example
> > iterating the loop:
> > aligned_size(A) <= buf_size && aligned_size(A+1) >= buf_size
> > => choose A
> > 
> > and may be also useful to be able to override the alignment in case of
> > need (working on a patch for that).
> > 
> > But apart from this I see more failures with other formats:
> > ffmpeg -i tests/lena.pnm -s 257x257 -vcodec rawvideo -pix_fmt PIX_FMT -y out.avi
> > 
> > in particular with formats monob, rgb565, rgb24, still investigating
> > (hints are welcome).
> > -- 
> > FFmpeg = Frenzy & Fascinating Monstrous Prodigious Elastic Ghost
> 
> >  rawdec.c |   23 +++++++++++++++--------
> >  1 file changed, 15 insertions(+), 8 deletions(-)
> > c02246ed0069dbe581c989237cc382e919c87a20  0004-lavc-rawdec-fix-palette-alignment-setting.patch
> > From dc6fd4caa5b7ee4925845785fab67a697f24aeb0 Mon Sep 17 00:00:00 2001
> > From: Stefano Sabatini <stefasab at gmail.com>
> > Date: Sat, 30 Jun 2012 19:50:48 +0200
> > Subject: [PATCH] lavc/rawdec: fix palette alignment setting
> > 
> > ---
> >  libavcodec/rawdec.c |   23 +++++++++++++++--------
> >  1 files changed, 15 insertions(+), 8 deletions(-)
> > 
> > diff --git a/libavcodec/rawdec.c b/libavcodec/rawdec.c
> > index a0211b6..91135fc 100644
> > --- a/libavcodec/rawdec.c
> > +++ b/libavcodec/rawdec.c
> > @@ -217,15 +217,22 @@ static int raw_decode(AVCodecContext *avctx,
> >              frame->palette_has_changed = 1;
> >          }
> >      }
> > -    if((avctx->pix_fmt==PIX_FMT_BGR24    ||
> > -        avctx->pix_fmt==PIX_FMT_GRAY8    ||
> > -        avctx->pix_fmt==PIX_FMT_RGB555LE ||
> > -        avctx->pix_fmt==PIX_FMT_RGB555BE ||
> > -        avctx->pix_fmt==PIX_FMT_RGB565LE ||
> > +    if (avctx->pix_fmt==PIX_FMT_BGR24     ||
> > +        avctx->pix_fmt==PIX_FMT_GRAY8     ||
> > +        avctx->pix_fmt==PIX_FMT_RGB555LE  ||
> > +        avctx->pix_fmt==PIX_FMT_RGB555BE  ||
> > +        avctx->pix_fmt==PIX_FMT_RGB565LE  ||
> >          avctx->pix_fmt==PIX_FMT_MONOWHITE ||
> > -        avctx->pix_fmt==PIX_FMT_PAL8) &&
> > -        FFALIGN(frame->linesize[0], linesize_align)*avctx->height <= buf_size)
> > -        frame->linesize[0] = FFALIGN(frame->linesize[0], linesize_align);
> > +        avctx->pix_fmt==PIX_FMT_PAL8) {
> > +        int aligned_plane_size = FFALIGN(frame->linesize[0], linesize_align)*avctx->height;
> > +        int plane_buf_size = avctx->pix_fmt == PIX_FMT_PAL8 && !context->buffer ?
> > +            buf_size - AVPALETTE_SIZE : buf_size;
> > +        if (aligned_plane_size < plane_buf_size) {
>                                  ^^^
> > +            av_log(avctx, AV_LOG_DEBUG, "aligned plane size %d <= plane buffer size %d, assuming alignment of %d\n",
>                                                                  ^^^^
> 
> mismatch
> 
> otherwise if it works it should be ok

ping

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The greatest way to live with honor in this world is to be what we pretend
to be. -- Socrates
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20130614/410c6df0/attachment.asc>