[FFmpeg-devel] [PATCH] lavc/videotoolbox: Consistent fallback to external software encoding/decoding.

[Thilo Borgmann]

Am 20.03.16 um 17:22 schrieb Richard Kern:
> The encoder isn’t meant to be a hwaccel - it uses hardware encoding when
> available, but I think it should fallback to software in the default case
> when hardware isn’t available.
> 
> One use case of this encoder is using a device-side H.264 encoder, so it
> doesn’t matter if it’s software or hardware.

Missed that - I agree.


> How about a flag or option to require hardware? 

I think knowing about it should be sufficient. Added a warning.

Updated patches attached.

-Thilo
-------------- next part --------------
From 2a4706cbe835c620ee7407c11f45f74aef6374cf Mon Sep 17 00:00:00 2001
From: Thilo Borgmann <thilo.borgmann at mail.de>
Date: Mon, 21 Mar 2016 11:23:30 +0100
Subject: [PATCH 1/2] lavc/videotoolbox: Fallback to external software decoding
 by codec flag.

    Fallback to external software decoding on user demand
    by specifying codec flag ext_sw_fallback.

    Fixes ticket #5352.
---
 libavcodec/avcodec.h       | 4 ++++
 libavcodec/options_table.h | 1 +
 libavcodec/videotoolbox.c  | 8 +++++++-
 3 files changed, 12 insertions(+), 1 deletion(-)

diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
index 637984b..c46400a 100644
--- a/libavcodec/avcodec.h
+++ b/libavcodec/avcodec.h
@@ -744,6 +744,10 @@ typedef struct RcOverride{
  */
 #define AV_CODEC_FLAG_QPEL            (1 <<  4)
 /**
+ * Allow external software fallback for external hardware accellerators
+ */
+#define AV_CODEC_FLAG_EXT_SW_FALLBACK (1 <<  5)
+/**
  * Use internal 2pass ratecontrol in first pass mode.
  */
 #define AV_CODEC_FLAG_PASS1           (1 <<  9)
diff --git a/libavcodec/options_table.h b/libavcodec/options_table.h
index d29996f..04a1e4e 100644
--- a/libavcodec/options_table.h
+++ b/libavcodec/options_table.h
@@ -52,6 +52,7 @@ static const AVOption avcodec_options[] = {
 {"unaligned", "allow decoders to produce unaligned output", 0, AV_OPT_TYPE_CONST, { .i64 = AV_CODEC_FLAG_UNALIGNED }, INT_MIN, INT_MAX, V | D, "flags" },
 {"mv4", "use four motion vectors per macroblock (MPEG-4)", 0, AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_FLAG_4MV }, INT_MIN, INT_MAX, V|E, "flags"},
 {"qpel", "use 1/4-pel motion compensation", 0, AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_FLAG_QPEL }, INT_MIN, INT_MAX, V|E, "flags"},
+{"ext_sw_fallback", "allow external software fallback for external hardware accellerators", 0, AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_FLAG_EXT_SW_FALLBACK }, INT_MIN, INT_MAX, V|E|D, "flags"},
 {"loop", "use loop filter", 0, AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_FLAG_LOOP_FILTER }, INT_MIN, INT_MAX, V|E, "flags"},
 {"qscale", "use fixed qscale", 0, AV_OPT_TYPE_CONST, {.i64 = AV_CODEC_FLAG_QSCALE }, INT_MIN, INT_MAX, 0, "flags"},
 #if FF_API_GMC
diff --git a/libavcodec/videotoolbox.c b/libavcodec/videotoolbox.c
index 2f4d531..8554f02 100644
--- a/libavcodec/videotoolbox.c
+++ b/libavcodec/videotoolbox.c
@@ -32,6 +32,10 @@
 #include "h264.h"
 #include "mpegvideo.h"
 
+#ifndef kVTVideoDecoderSpecification_EnableHardwareAcceleratedVideoDecoder
+#  define kVTVideoDecoderSpecification_EnableHardwareAcceleratedVideoDecoder CFSTR("EnableHardwareAcceleratedVideoDecoder")
+#endif
+
 #ifndef kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder
 #  define kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder CFSTR("RequireHardwareAcceleratedVideoDecoder")
 #endif
@@ -412,7 +416,9 @@ static CFDictionaryRef videotoolbox_decoder_config_create(CMVideoCodecType codec
                                                                    &kCFTypeDictionaryValueCallBacks);
 
     CFDictionarySetValue(config_info,
-                         kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder,
+                         (avctx->flags & AV_CODEC_FLAG_EXT_SW_FALLBACK) ?
+                            kVTVideoDecoderSpecification_EnableHardwareAcceleratedVideoDecoder :
+                            kVTVideoDecoderSpecification_RequireHardwareAcceleratedVideoDecoder,
                          kCFBooleanTrue);
 
     if (avctx->extradata_size) {
-- 
2.5.4 (Apple Git-61)

-------------- next part --------------
From c1f1fd4b60b5fdf13e6e64f999646ee94f9bf819 Mon Sep 17 00:00:00 2001
From: Thilo Borgmann <thilo.borgmann at mail.de>
Date: Mon, 21 Mar 2016 11:24:43 +0100
Subject: [PATCH 2/2] lavc/videotoolboxenc: Print warning if falling back to
 software encoder.

---
 libavcodec/videotoolboxenc.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/libavcodec/videotoolboxenc.c b/libavcodec/videotoolboxenc.c
index 3ed1f64..c5e0463 100644
--- a/libavcodec/videotoolboxenc.c
+++ b/libavcodec/videotoolboxenc.c
@@ -511,6 +511,8 @@ static av_cold int vtenc_init(AVCodecContext *avctx)
 
 #if !TARGET_OS_IPHONE
     if (status != 0 || !vtctx->session) {
+        av_log(avctx, AV_LOG_WARNING, "Creating hardware encoder failed. Using software encoder.\n");
+
         CFDictionaryRemoveValue(enc_info, kVTVideoEncoderSpecification_RequireHardwareAcceleratedVideoEncoder);
 
         status = VTCompressionSessionCreate(
-- 
2.5.4 (Apple Git-61)