[FFmpeg-devel] [PATCH 2/2] lavu: relax the condition to do hwframe unmapping.

[Ruiling Song]

This patch aims to fix failure of hwmap/hwunmap working against
passthrough mode filters like with transpose_opencl:
[vaapi_frame] "hwmap,transpose_opencl=passthrough=landscape,
hwmap=derive_device=vaapi:reverse=1" [vappi_frame]

If the frame meet the pass-through criteria, then the output of the
first hwmap will directly goes into the input of the second hwmap.
What we need to do here is simply unmap the frame. The current issue
is when we try to do unmap in the frame-context of the second hwmap,
it fails to meet the check in av_hwframe_map(), which requires the
original hw_frames_ctx same as the destination hw_frames_ctx. But I
think that if we are trying to map to the same device as the orginal
device_ctx, then we can just do the unmap.

Signed-off-by: Ruiling Song <ruiling.song at intel.com>
---
I am not sure if there are any concern or side-effects of doing like this?
The first idea came up to fix the issue is do the checking against
internal->source_frames in vf_hwmap.c. but I find that this is not accessible
outside libavutil. So I use this fix. Hope to have your comment and discussion.

Ruiling

 libavutil/hwcontext.c | 9 +++++----
 libavutil/hwcontext.h | 6 +++---
 2 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/libavutil/hwcontext.c b/libavutil/hwcontext.c
index f1e404a..a006212 100644
--- a/libavutil/hwcontext.c
+++ b/libavutil/hwcontext.c
@@ -739,20 +739,21 @@ fail:
 
 int av_hwframe_map(AVFrame *dst, const AVFrame *src, int flags)
 {
-    AVHWFramesContext *src_frames, *dst_frames;
+    AVHWFramesContext *src_frames, *dst_frames, *src_src = NULL;
     HWMapDescriptor *hwmap;
     int ret;
 
     if (src->hw_frames_ctx && dst->hw_frames_ctx) {
         src_frames = (AVHWFramesContext*)src->hw_frames_ctx->data;
         dst_frames = (AVHWFramesContext*)dst->hw_frames_ctx->data;
+        if (src_frames->internal->source_frames)
+            src_src =
+                (AVHWFramesContext*)src_frames->internal->source_frames->data;
 
         if ((src_frames == dst_frames &&
              src->format == dst_frames->sw_format &&
              dst->format == dst_frames->format) ||
-            (src_frames->internal->source_frames &&
-             src_frames->internal->source_frames->data ==
-             (uint8_t*)dst_frames)) {
+            (src_src && src_src->device_ctx == dst_frames->device_ctx)) {
             // This is an unmap operation.  We don't need to directly
             // do anything here other than fill in the original frame,
             // because the real unmap will be invoked when the last
diff --git a/libavutil/hwcontext.h b/libavutil/hwcontext.h
index f5a4b62..efe3988 100644
--- a/libavutil/hwcontext.h
+++ b/libavutil/hwcontext.h
@@ -528,9 +528,9 @@ enum {
  * by av_frame_alloc()).  src should have an associated hwframe context, and
  * dst may optionally have a format and associated hwframe context.
  *
- * If src was created by mapping a frame from the hwframe context of dst,
- * then this function undoes the mapping - dst is replaced by a reference to
- * the frame that src was originally mapped from.
+ * If src was created by mapping a frame from a hwframe context which shares the
+ * same device_ctx with dst, then this function undoes the mapping - dst is
+ * replaced by a reference to the frame that src was originally mapped from.
  *
  * If both src and dst have an associated hwframe context, then this function
  * attempts to map the src frame from its hardware context to that of dst and
-- 
2.7.4