[FFmpeg-devel] [PATCH] avutil/hwcontext_dxva2: Don't improperly free IDirect3DSurface9 objects

[wm4]

On Tue, 16 May 2017 05:04:36 -0700
Aaron Levinson <alevinsn at aracnet.com> wrote:

> Add dxva2_pool_release_dummy() and use it in call to
> av_buffer_create() in dxva2_pool_alloc().
> 
> Prior to this change, av_buffer_create() was called with NULL for the
> third argument, which indicates that av_buffer_default_free() should
> be used to free the buffer's data.  Eventually, it gets to
> buffer_pool_free() and calls buf->free() on a surface object (which is
> av_buffer_default_free()).
> 
> This can result in a crash when the debug version of the C-runtime is
> used on Windows.  While it doesn't appear to result in a crash when
> the release version of the C-runtime is used on Windows, it likely
> results in memory corruption, since av_free() is being called on
> memory that was allocated using
> IDirectXVideoAccelerationService::CreateSurface().
> 
> Signed-off-by: Aaron Levinson <alevinsn at aracnet.com>
> ---
>  libavutil/hwcontext_dxva2.c | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/libavutil/hwcontext_dxva2.c b/libavutil/hwcontext_dxva2.c
> index 4ed0d56..6c41788 100644
> --- a/libavutil/hwcontext_dxva2.c
> +++ b/libavutil/hwcontext_dxva2.c
> @@ -121,6 +121,13 @@ static void dxva2_frames_uninit(AVHWFramesContext *ctx)
>      }
>  }
>  
> +static void dxva2_pool_release_dummy(void *opaque, uint8_t *data)
> +{
> +    // important not to free anything here--data is a surface object
> +    // associated with the call to CreateSurface(), and these surfaces are
> +    // released in dxva2_frames_uninit()
> +}
> +
>  static AVBufferRef *dxva2_pool_alloc(void *opaque, int size)
>  {
>      AVHWFramesContext      *ctx = (AVHWFramesContext*)opaque;
> @@ -130,7 +137,7 @@ static AVBufferRef *dxva2_pool_alloc(void *opaque, int size)
>      if (s->nb_surfaces_used < hwctx->nb_surfaces) {
>          s->nb_surfaces_used++;
>          return av_buffer_create((uint8_t*)s->surfaces_internal[s->nb_surfaces_used - 1],
> -                                sizeof(*hwctx->surfaces), NULL, 0, 0);
> +                                sizeof(*hwctx->surfaces), dxva2_pool_release_dummy, 0, 0);
>      }
>  
>      return NULL;

LGTM