[FFmpeg-devel] Memory leak using bitstream filters with shared libs

[Rich Felker]

On Sun, Mar 09, 2008 at 12:05:33AM +0000, M?ns Rullg?rd wrote:
> Baptiste discovered a rather nasty memory leak when using a bitstream
> filter with shared libraries.
> 
> In ffmpeg.c:write_frame() the output of bitstream filters is stored in
> an AVPacket, and its destruct function is set like this (line 417):
> 
>             new_pkt.destruct= av_destruct_packet;
> 
> Later on, in av_interleave_packet_per_dts(), AVPacket.destruct is
> compared against av_destruct_packet, like this (utils.c:2439):
> 
>         if(pkt->destruct == av_destruct_packet)
> 
> The trouble here is that with shared libraries, the address of
> av_destruct_packet when used in the main executable or libs other than
> the one defining it resolves to a trampoline that calls the real
> function using a GOT entry.  As a result, this comparison is always
> false when the destruct pointer was assigned outside libavformat.
> 
> Does anyone have an idea for fixing this?

It's actually an implementation bug since the C language requires that
the addresses match. I suspect it's binutils doing something stupid
due to -Bsymbolic. Anyway this sort of problem is common enough that
I'd recommend avoiding comparison of function pointers. I'm not
familiar enough with the code to suggest an alternative though.

Rich