[FFmpeg-devel] Memory leak using bitstream filters with shared libs

[Måns Rullgård]

Rich Felker <dalias at aerifal.cx> writes:

> On Sun, Mar 09, 2008 at 12:05:33AM +0000, M?ns Rullg?rd wrote:
>> Baptiste discovered a rather nasty memory leak when using a bitstream
>> filter with shared libraries.
>> 
>> In ffmpeg.c:write_frame() the output of bitstream filters is stored in
>> an AVPacket, and its destruct function is set like this (line 417):
>> 
>>             new_pkt.destruct= av_destruct_packet;
>> 
>> Later on, in av_interleave_packet_per_dts(), AVPacket.destruct is
>> compared against av_destruct_packet, like this (utils.c:2439):
>> 
>>         if(pkt->destruct == av_destruct_packet)
>> 
>> The trouble here is that with shared libraries, the address of
>> av_destruct_packet when used in the main executable or libs other than
>> the one defining it resolves to a trampoline that calls the real
>> function using a GOT entry.  As a result, this comparison is always
>> false when the destruct pointer was assigned outside libavformat.
>> 
>> Does anyone have an idea for fixing this?
>
> It's actually an implementation bug since the C language requires that
> the addresses match.

I was hoping you'd say that.

> I suspect it's binutils doing something stupid due to -Bsymbolic.

That is my theory as well, though I've not confirmed it.

> Anyway this sort of problem is common enough that I'd recommend
> avoiding comparison of function pointers. I'm not familiar enough
> with the code to suggest an alternative though.

I dread to think what might happen on Windows.

-- 
M?ns Rullg?rd
mans at mansr.com