[FFmpeg-devel] [Patch] parser.c in libavcodec

[Michael Niedermayer]

On Sun, Mar 23, 2008 at 03:56:54PM -0400, andy g wrote:
> It seems that the return value of av_fast_realloc in parser.c is never 
> checked - so if the allocation fails for any reason, the function will 
> proceed normally. I have added a couple of statements in this patch to 
> prevent this from happening. I am a little unclear on the post-conditions 
> that the ff_combine_frame function provides, so I simply set the 
> buffer_size to 0 (since the buffer is null if the allocation fails) and 
> return -1. This is the cause of a series of SIGSEGVs in the MLP patch - 
> which unfortunately I now require Ian's help on again. After running his 
> code with this patch, I noticed that he often ignores the response of 
> ff_combine_frame in almost every calling of mlp_parse which leads to a 
> segfault in some cases. I would fix this, however the function does not 
> provide comments and I am unsure of what the correct response is to a 
> failure of ff_combine_frame in these instances.

The patch you attached contains no buffer_size, no ff_combine_frame() and
no av_fast_realloc().

The return value of av_fast_realloc() should of course be checked. But if
you do dont forget that *realloc() does not free the original buffer on
failure.
About the correct behavior in case of such failures, well almost 
anything that doesnt crash and is not exploitable should do. Simply
freeing the buffer and updating related variables for example.
About the return of ff_combine_frame()
return AVERROR(ENOMEM) and update the doxy above ff_combine_frame()

And no i didnt expect you to find a bug outside MLP, but iam happy about
every found and fixed bug.


[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

I have often repented speaking, but never of holding my tongue.
-- Xenocrates
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20080323/d4da31c6/attachment.pgp>