[FFmpeg-devel] [PATCH] Do not access beyond the end of the palette in pcxenc

[Michael Niedermayer]

On Tue, May 12, 2009 at 09:46:42PM -0400, Daniel Verkamp wrote:
> Hi,
> 
> Attached is a patch to fix the monoblack case of pcxenc; it was
> accessing beyond the end of the array.
> 
> An alternate fix (second patch) would involve just making
> monoblack_pal 16 entries long instead of 2.
> 
> I don't have any real preference for one patch over the other, but the
> code for the second fix stays slightly cleaner, at the expense of data
> size vs. code size.

i ve no real preferance either


> 
> (As an aside, it seems to me that AVFrame.data[1] could be set with a
> black and white palette for monoblack/monowhite pixfmts to avoid
> duplication of this kind of special-case code in codecs.)

i agree


> 
> Thanks,
> -- Daniel Verkamp

> From 9a9608206d3989d881218555d30e83c30a20a9f6 Mon Sep 17 00:00:00 2001
> From: Daniel Verkamp <daniel at drv.nu>
> Date: Tue, 12 May 2009 20:43:13 -0500
> Subject: [PATCH] Do not access beyond the end of the palette
> 
> ---
>  libavcodec/pcxenc.c |    8 ++++++--
>  1 files changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/libavcodec/pcxenc.c b/libavcodec/pcxenc.c
> index 36f7d1d..ea51774 100644
> --- a/libavcodec/pcxenc.c
> +++ b/libavcodec/pcxenc.c
> @@ -154,8 +154,12 @@ static int pcx_encode_frame(AVCodecContext *avctx,
>      bytestream_put_le16(&buf, avctx->height - 1);   // y max
>      bytestream_put_le16(&buf, 0);                   // horizontal DPI
>      bytestream_put_le16(&buf, 0);                   // vertical DPI
> -    for (i = 0; i < 16; i++)
> -        bytestream_put_be24(&buf, pal ? pal[i] : 0);// palette (<= 16 color only)
> +    i = 0;
> +    if (pal)
> +        for (; i < FFMIN(1 << (bpp * nplanes), 16); i++)
> +            bytestream_put_be24(&buf, pal[i]);      // palette (<= 16 color only)
> +    for (; i < 16; i++)
> +        bytestream_put_be24(&buf, 0);

pal && i < (1<<bpp * nplanes)


[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Good people do not need laws to tell them to act responsibly, while bad
people will find a way around the laws. -- Plato
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20090513/16cc3467/attachment.pgp>