[FFmpeg-devel] [PATCH] fix the SAMPLE_FMT_NONE case in ffmdec.c

[Peter Ross]

On Mon, Mar 15, 2010 at 04:15:10PM -0400, Ronald S. Bultje wrote:
> Hi,
> 
> see attached. right now, SAMPLE_FMT_NONE results in the invalid value 65535.
> 
> Another (possibly better) fix is to change the value to 32-bit (as is
> done with pix_fmt - that's why PIX_FMT_NONE is transmitted just fine,
> but SAMPLE_FMT_NONE is not), but that would change the actual format.
> 
> Ronald

> Index: libavformat/ffmdec.c
> ===================================================================
> --- libavformat/ffmdec.c	(revision 22550)
> +++ libavformat/ffmdec.c	(working copy)
> @@ -355,6 +355,8 @@
>              codec->channels = get_le16(pb);
>              codec->frame_size = get_le16(pb);

>              codec->sample_fmt = get_le16(pb);
> +            if (codec->sample_fmt == (1 << 16) - 1)
> +                codec->sample_fmt = -1;

why not read it as a signed 16-bit integer? e.g.
	codec->sample_fmt = sign_extend( get_le16(), 16 )

the fact that we dont need to do this with pix_fmt is just a fluke.
if sizeof(int) was > 32-bits the same problem would occur there too.

-- Peter
(A907 E02F A6E5 0CD2 34CD 20D2 6760 79C5 AC40 DD6B)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20100316/73dced55/attachment.pgp>