[MPlayer-dev-eng] Serious memory exhaustion when playing H264 by ffh264vda decoder

[Xidorn Quan]

On Mon, Nov 18, 2013 at 8:23 AM, Zongyao Qu <zongyao.qu at gmail.com> wrote:
> I found the following patch could fix this issue.

This patch is very helpful to locate the problem, thanks.

The problem is that, h264_vda in ffmpeg is a special decoder which
must manage the buffer itself since it is in fact a wrapper decoder of
h264. But the code which is removed in the patch provided by Zongyao
explicitly overrides the buffer management functions set during
initialization of h264_vda.

@reimar, I reviewed your commit r36422, the one mentioned by Zongyao.
I found that such code (overrides buffer functions when do_dr1 is
false) didn't exist in the old code at all, so I think there might be
some consideration. What did you think then? Besides, I think we have
to avoid any change in buffer functions (and any parameters might have
been used by avcodec) after avcodec_open2. I don't have an idea what
to do now since the code in vd_ffmpeg seems to be fragile. Any
suggestion?