[FFmpeg-devel] [PATCH] prevent buffer overflow with large a/mulaw frames

[Peter Ross]

On Sun, Jul 26, 2009 at 03:32:59PM +1000, Peter Ross wrote:
> On Sat, Jul 25, 2009 at 09:42:52PM -0700, Baptiste Coudurier wrote:
> > Hi Peter,
> >
> > On 07/25/2009 09:19 PM, Peter Ross wrote:
> >> Hi,
> >>
> >> This patch prevents alaw/mulaw decoders from writing beyond the output buffer.
> >>
> 
> > I think output buffer size is stored in *data_size.
> > Code should check against this, but it seems it is already. Is the check  
> > wrong ?
> >
> > Code is:
> >     buf_size= FFMIN(buf_size, *data_size/2);
> >     *data_size=0;
> >
> >     n = buf_size/sample_size;
> 
> You are correct, the bug actually exists in the *encoder* where there is no
> such constraint on n. Updated patch enclosed.

Actually... this bug originates within ffmpeg. The audio output buf_size passed
to codec->encode does NOT reflect the size of the samples buffer.

Per suggestion from Baptiste, I have modified ffmpeg.c to auto realloc the
encoder buffer. Please disregard the previous libavcodec/pcm.c diffs in this
email thread.

-- Peter
(A907 E02F A6E5 0CD2 34CD 20D2 6760 79C5 AC40 DD6B)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ffmpeg-audiooutsize.diff
Type: text/x-diff
Size: 1702 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20090726/d7598f58/attachment.diff>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20090726/d7598f58/attachment.pgp>