[FFmpeg-cvslog] avutil/hwcontext_dxva2: Don't improperly free IDirect3DSurface9 objects

[Aaron Levinson]

ffmpeg | branch: master | Aaron Levinson <alevinsn at aracnet.com> | Tue May 16 14:23:27 2017 -0700| [3d040513a1de4797a4f81dde4984395f51db76b7] | committer: Anton Khirnov

avutil/hwcontext_dxva2: Don't improperly free IDirect3DSurface9 objects

Add dxva2_pool_release_dummy() and use it in call to
av_buffer_create() in dxva2_pool_alloc().

Prior to this change, av_buffer_create() was called with NULL for the
third argument, which indicates that av_buffer_default_free() should
be used to free the buffer's data.  Eventually, it gets to
buffer_pool_free() and calls buf->free() on a surface object (which is
av_buffer_default_free()).

This can result in a crash when the debug version of the C-runtime is
used on Windows.  While it doesn't appear to result in a crash when
the release version of the C-runtime is used on Windows, it likely
results in memory corruption, since av_free() is being called on
memory that was allocated using
IDirectXVideoAccelerationService::CreateSurface().

Signed-off-by: Aaron Levinson <alevinsn at aracnet.com>
Reviewed-by: wm4 <nfxjfg at googlemail.com>
Reviewed-by: Steven Liu <lingjiujianke at gmail.com>
Reviewed-by: Mark Thompson <sw at jkqxz.net>
Signed-off-by: Anton Khirnov <anton at khirnov.net>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=3d040513a1de4797a4f81dde4984395f51db76b7
---

 libavutil/hwcontext_dxva2.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/libavutil/hwcontext_dxva2.c b/libavutil/hwcontext_dxva2.c
index 3790bed4b7..d1b19ab237 100644
--- a/libavutil/hwcontext_dxva2.c
+++ b/libavutil/hwcontext_dxva2.c
@@ -119,6 +119,13 @@ static void dxva2_frames_uninit(AVHWFramesContext *ctx)
     }
 }
 
+static void dxva2_pool_release_dummy(void *opaque, uint8_t *data)
+{
+    // important not to free anything here--data is a surface object
+    // associated with the call to CreateSurface(), and these surfaces are
+    // released in dxva2_frames_uninit()
+}
+
 static AVBufferRef *dxva2_pool_alloc(void *opaque, int size)
 {
     AVHWFramesContext      *ctx = (AVHWFramesContext*)opaque;
@@ -128,7 +135,7 @@ static AVBufferRef *dxva2_pool_alloc(void *opaque, int size)
     if (s->nb_surfaces_used < hwctx->nb_surfaces) {
         s->nb_surfaces_used++;
         return av_buffer_create((uint8_t*)s->surfaces_internal[s->nb_surfaces_used - 1],
-                                sizeof(*hwctx->surfaces), NULL, 0, 0);
+                                sizeof(*hwctx->surfaces), dxva2_pool_release_dummy, 0, 0);
     }
 
     return NULL;