[FFmpeg-devel] [PATCH v2] avfilter: add libdewobble_opencl filter
Paul B Mahol
onemda at gmail.com
Mon Aug 23 20:09:12 EEST 2021
On Mon, Aug 23, 2021 at 4:18 PM Daniel Playfair Cal <
daniel.playfair.cal at gmail.com> wrote:
> All of the existing filters for video stabilization use an affine model
> (or a limited version of it) to represent the movement of the camera. When
> used with cameras with a very wide field of view and/or where the camera
> shaking is severe, the corrections result in significant geometric
> distortion ("wobbling").
>
> Dewobble (https://git.sr.ht/~hedgepigdaniel/dewobble) is a library built
> to solve this problem. It requires knowledge of the projection used by
> the input camera, and it performs stabilization using a homography
> model, which is limited to include only changes in camera orientation.
> Additionally, it can perform projection change by specifying a different
> projection for the output camera. This is more efficient and results in
> less loss of information than using separate filters to perform
> stabilization and projection change.
>
> The dewobble_opencl filter is a wrapper for Dewobble. Dewobble supports
> input and output in OpenCL buffers containing NV12 frames. Hence, the
> filter is named dewobble_opencl and has the same limitations. Currently
> all of the options of Dewobble are supported. Of the two types of filter
> available in Dewobble (FilterSync and FilterThreaded), FilterThreaded is
> used. The API is synchronous, but the transformations are done in a
> separate thread. The purpose of this is to isolate the global per thread
> OpenCL context used by OpenCV, which Dewobble uses internally. This
> prevents dewobble_opencl from interfering with any other usage of OpenCV
> from within FFmpeg.
>
> Signed-off-by: Daniel Playfair Cal <daniel.playfair.cal at gmail.com>
> ---
>
> Changelog v2:
> - style improvements
> - rename from "dewobble_opencl" to "libdewobble_opencl"
>
library is named dewobble, thus filter should be libdewobble.
>
> I'm still confused as to why this filter should be prefixed with lib but
> not others that wrap external libraries like lensfun, vidstabtransform,
> vidstabdetect, etc. In any case, I've renamed as you requested.
>
> I've addressed all the comments about style as well as doing a general
> cleanup. Function arguments are arranged in a different way which
> doesn't result in so many new lines.
>
> ---
> Changelog | 1 +
> LICENSE.md | 2 +-
> configure | 4 +
> doc/filters.texi | 149 ++++
> libavfilter/Makefile | 1 +
> libavfilter/allfilters.c | 1 +
> libavfilter/version.h | 2 +-
> libavfilter/vf_libdewobble_opencl.c | 1273 +++++++++++++++++++++++++++
> 8 files changed, 1431 insertions(+), 2 deletions(-)
> create mode 100644 libavfilter/vf_libdewobble_opencl.c
>
> diff --git a/Changelog b/Changelog
> index 5a5b50eb66..a8d71ab4ee 100644
> --- a/Changelog
> +++ b/Changelog
> @@ -11,6 +11,7 @@ version <next>:
> - afwtdn audio filter
> - audio and video segment filters
> - Apple Graphics (SMC) encoder
> +- Dewobble filter
>
no, libdewobble video filter
>
>
> version 4.4:
> diff --git a/LICENSE.md b/LICENSE.md
> index 613070e1b6..dfdf010d8e 100644
> --- a/LICENSE.md
> +++ b/LICENSE.md
> @@ -112,7 +112,7 @@ The VMAF, mbedTLS, RK MPI, OpenCORE and VisualOn
> libraries are under the Apache
> version 3 of those licenses. So to combine these libraries with FFmpeg,
> the
> license version needs to be upgraded by passing `--enable-version3` to
> configure.
>
> -The smbclient library is under the GPL v3, to combine it with FFmpeg,
> +The dewobble and smbclient libraries are under the GPL v3, to combine
> them with FFmpeg,
> the options `--enable-gpl` and `--enable-version3` have to be passed to
> configure to upgrade FFmpeg to the GPL v3.
>
> diff --git a/configure b/configure
> index 9249254b70..60b3d3dbea 100755
> --- a/configure
> +++ b/configure
> @@ -230,6 +230,7 @@ External library support:
> --enable-libdavs2 enable AVS2 decoding via libdavs2 [no]
> --enable-libdc1394 enable IIDC-1394 grabbing using libdc1394
> and libraw1394 [no]
> + --enable-libdewobble enable video stabilization via libdewobble [no]
> --enable-libfdk-aac enable AAC de/encoding via libfdk-aac [no]
> --enable-libflite enable flite (voice synthesis) support via
> libflite [no]
> --enable-libfontconfig enable libfontconfig, useful for drawtext
> filter [no]
> @@ -1781,6 +1782,7 @@ EXTERNAL_LIBRARY_VERSION3_LIST="
> "
>
> EXTERNAL_LIBRARY_GPLV3_LIST="
> + libdewobble
> libsmbclient
> "
>
> @@ -3606,6 +3608,7 @@ interlace_filter_deps="gpl"
> kerndeint_filter_deps="gpl"
> ladspa_filter_deps="ladspa libdl"
> lensfun_filter_deps="liblensfun version3"
> +libdewobble_opencl_filter_deps="libdewobble opencl"
> lv2_filter_deps="lv2"
> mcdeint_filter_deps="avcodec gpl"
> metadata_filter_deps="avformat"
> @@ -6406,6 +6409,7 @@ enabled libcodec2 && require libcodec2
> codec2/codec2.h codec2_create -lc
> enabled libdav1d && require_pkg_config libdav1d "dav1d >= 0.5.0"
> "dav1d/dav1d.h" dav1d_version
> enabled libdavs2 && require_pkg_config libdavs2 "davs2 >= 1.6.0"
> davs2.h davs2_decoder_open
> enabled libdc1394 && require_pkg_config libdc1394 libdc1394-2
> dc1394/dc1394.h dc1394_new
> +enabled libdewobble && require_pkg_config libdewobble dewobble
> dewobble/filter.h dewobble_filter_create_threaded
> enabled libdrm && require_pkg_config libdrm libdrm xf86drm.h
> drmGetVersion
> enabled libfdk_aac && { check_pkg_config libfdk_aac fdk-aac
> "fdk-aac/aacenc_lib.h" aacEncOpen ||
> { require libfdk_aac fdk-aac/aacenc_lib.h
> aacEncOpen -lfdk-aac &&
> diff --git a/doc/filters.texi b/doc/filters.texi
> index c84202cf85..f8f6528479 100644
> --- a/doc/filters.texi
> +++ b/doc/filters.texi
> @@ -14129,6 +14129,155 @@ ffmpeg -i input.mov -vf
> lensfun=make=Canon:model="Canon EOS 100D":lens_model="Ca
>
> @end itemize
>
> + at section libdewobble_opencl
> +
> +Apply motion stabilization with awareness of lens projection and/or lens
> projection change using libdewobble (@url{
> https://git.sr.ht/~hedgepigdaniel/dewobble}).
> +
> +To enable compilation of this filter you need to configure FFmpeg with
> + at code{--enable-libdewobble}.
> +
> +This filter accepts the following options:
> +
> + at table @option
> + at item in_p
> + at item out_p
> +Set the lens projection model for the input and output.
> +
> +Available values are:
> + at table @samp
> + at item rect
> +Rectilinear projection.
> +
> + at item fish
> +Equidistant fisheye projection.
> +
> + at end table
> +
> + at item in_dfov
> + at item out_dfov
> +Diagonal field of view in degrees for the input and output.
> +
> + at item in_fx
> + at item in_fy
> + at item out_fx
> + at item out_fy
> +Location of the focal point in the input and output image.
> +Default value is the image centre in both cases.
> +
> + at item out_w
> + at item out_h
> +Dimensions of the output image.
> +Default value is the same as in input image.
> +
> + at item stab
> +Motion stabilization algorithm.
> +
> +Available values are:
> + at table @samp
> + at item fixed
> +Fix the camera orientation after the first frame.
> +
> + at item none
> +No not apply stabilization.
> +
> + at item sg
> +Smooth the camera motion using a Savitzky-Golay filter.
> +
> + at end table
> +
> +Default value is @samp{sg}.
> +
> + at item stab_r
> +For Savitzky-Golay smoothing: the number of frames to look ahead and
> behind.
> +Higher values result in a smoother output camera path.
> +
> +Default value is 15.
> +
> +Higher values increase (OpenCL) memory usage.
> +
> + at item stab_h
> +For stabilization: the number of frames to look ahead to interpolate
> input camera rotation in frames where it cannot be detected.
> +
> +Default value is 30.
> +
> +Higher values increase (OpenCL) memory usage.
> +
> + at item interp
> +Pixel interpolation algorithm.
> +
> +Available values are:
> + at table @samp
> + at item nearest
> +Nearest neighbour interpolation (fast OpenCL implementation).
> +
> + at item linear
> +Bilinear interpolation (fast OpenCL implementation).
> +
> + at item cubic
> +Bicubic interpolation (CPU implementation).
> +
> + at item lanczos
> +Lanczos4 interpolation in an 8x8 neighbourhood (CPU implementation).
> +
> + at end table
> +
> +Default value is @samp{linear}.
> +
> + at item border
> +Border extrapolation algorithm (determines how to color pixels in the
> output that do not map to the input).
> +
> +Available values are:
> + at table @samp
> + at item constant
> +Constant color.
> +
> + at item reflect
> +Reflection of the input horizontally or vertically about the edge.
> +
> + at item reflect101
> +Reflection of the input horizontally or vertically about the point half a
> pixel from the edge.
> +
> + at item replicate
> +Replicate the pixel on the edge in a vertical or horizontal direction.
> +
> + at item wrap
> +Wrap around to the opposite side of the source image.
> +
> + at end table
> +
> +Default value is @samp{constant}.
> +
> + at item border_r
> + at item border_g
> + at item border_b
> +For @samp{constant} border, the color to fill with (red, green, blue
> components).
> +
> +Default value is black.
> +
> + at item debug
> +Include a suite of debugging information in the output.
> +
> +Default value is disabled.
> +
> + at end table
> +
> + at subsection Examples
> +
> + at itemize
> + at item
> +Apply motion stabilization to video from a popular action cam in a
> certain capture mode:
> + at example
> +ffmpeg -i INPUT -vf
> libdewobble_opencl=in_p=fish:in_dfov=145.8:out_p=fish:out_dfov=145.8:stab=sg
> OUTPUT
> + at end example
> +
> + at item
> +Apply stabilization and lens projection change:
> + at example
> +ffmpeg -i INPUT -vf
> libdewobble_opencl=in_p=fish:in_dfov=145.8:out_p=rect:out_dfov=145.8:stab=sg
> OUTPUT
> + at end example
> +
> + at end itemize
> +
> @section libvmaf
>
> Obtain the VMAF (Video Multi-Method Assessment Fusion)
> diff --git a/libavfilter/Makefile b/libavfilter/Makefile
> index 102ce7beff..c9399f8f68 100644
> --- a/libavfilter/Makefile
> +++ b/libavfilter/Makefile
> @@ -313,6 +313,7 @@ OBJS-$(CONFIG_KIRSCH_FILTER) +=
> vf_convolution.o
> OBJS-$(CONFIG_LAGFUN_FILTER) += vf_lagfun.o
> OBJS-$(CONFIG_LENSCORRECTION_FILTER) += vf_lenscorrection.o
> OBJS-$(CONFIG_LENSFUN_FILTER) += vf_lensfun.o
> +OBJS-$(CONFIG_LIBDEWOBBLE_OPENCL_FILTER) += vf_libdewobble_opencl.o
> opencl.o
> OBJS-$(CONFIG_LIBVMAF_FILTER) += vf_libvmaf.o framesync.o
> OBJS-$(CONFIG_LIMITER_FILTER) += vf_limiter.o
> OBJS-$(CONFIG_LOOP_FILTER) += f_loop.o
> diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
> index 73040d2824..95be7cb568 100644
> --- a/libavfilter/allfilters.c
> +++ b/libavfilter/allfilters.c
> @@ -298,6 +298,7 @@ extern const AVFilter ff_vf_kirsch;
> extern const AVFilter ff_vf_lagfun;
> extern const AVFilter ff_vf_lenscorrection;
> extern const AVFilter ff_vf_lensfun;
> +extern const AVFilter ff_vf_libdewobble_opencl;
> extern const AVFilter ff_vf_libvmaf;
> extern const AVFilter ff_vf_limiter;
> extern const AVFilter ff_vf_loop;
> diff --git a/libavfilter/version.h b/libavfilter/version.h
> index bcd27aa6e8..e9a76c5ac3 100644
> --- a/libavfilter/version.h
> +++ b/libavfilter/version.h
> @@ -30,7 +30,7 @@
> #include "libavutil/version.h"
>
> #define LIBAVFILTER_VERSION_MAJOR 8
> -#define LIBAVFILTER_VERSION_MINOR 3
> +#define LIBAVFILTER_VERSION_MINOR 4
> #define LIBAVFILTER_VERSION_MICRO 100
>
>
> diff --git a/libavfilter/vf_libdewobble_opencl.c
> b/libavfilter/vf_libdewobble_opencl.c
> new file mode 100644
> index 0000000000..74c2940877
> --- /dev/null
> +++ b/libavfilter/vf_libdewobble_opencl.c
> @@ -0,0 +1,1273 @@
> +/*
> + * Copyright (c) 2021 Daniel Playfair Cal <daniel.playfair.cal at gmail.com>
> + *
> + * This file is part of FFmpeg.
> + *
> + * This program is free software: you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation, either version 3 of the License, or
> + * (at your option) any later version.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <https://www.gnu.org/licenses/>.
> + */
> +#include <dewobble/camera.h>
> +#include <dewobble/filter.h>
> +#include <dewobble/stabilizer.h>
> +#include <float.h>
> +#include <pthread.h>
> +#include <signal.h>
> +
> +#include "libavutil/avassert.h"
> +#include "libavutil/common.h"
> +#include "libavutil/imgutils.h"
> +#include "libavutil/mem.h"
> +#include "libavutil/opt.h"
> +#include "libavutil/pixdesc.h"
> +#include "libavutil/thread.h"
> +
> +#include "avfilter.h"
> +#include "filters.h"
> +#include "internal.h"
> +#include "opencl.h"
> +#include "opencl_source.h"
> +#include "transpose.h"
> +#include "video.h"
> +
> +/**
> + * @file
> + * Apply motion stabilization with awareness of lens projection and/or
> change
> + * camera projection.
> + *
> + * This filter is essentially a wrapper around dewobble
> + * (https://git.sr.ht/~hedgepigdaniel/dewobble).
> + *
> + * @par Queued frames
> + *
> + * libdewobble requires a queue of frames before it can provide output
> because
> + * it looks ahead to calculate a smooth camera path and to interpolate
> camera
> + * positions from frames where it fails to detect motion. The number of
> queued
> + * frames required is determined by libdewobble.
> + *
> + * @par Hardware frame allocation
> + *
> + * Input OpenCL hardware frames contain `cl_image`s but these must be
> converted
> + * to `cl_buffer`s for libdewobble. Although the filter keeps a reference
> to
> + * the input frame until the output frame is sent, it unreferences the
> original
> + * hardware buffers immediately after copying them to a `cl_buffer` in
> + * `consume_input_frame`. This avoids OOM issues for example when using
> input
> + * frames mapped from VA-API hardware frames where there is a low limit
> for how
> + * many can be allocated at once. The filter only owns a single
> input/output
> + * hardware frame buffer at any time, although internally it allocates
> OpenCL
> + * buffers to store the contents of a queue of frames.
> + */
> +
> +/**
> + * Camera properties, mirroring those present in libdewobble's camera
> object.
> + */
> +typedef struct Camera {
> + /**
> + * Camera projection model, e.g. `DEWOBBLE_PROJECTION_RECTILINEAR`
> + */
> + int model;
> +
> + /**
> + * Camera diagonal field of view in degrees
> + */
> + double diagonal_fov;
> +
> + /**
> + * Width in pixels
> + */
> + int width;
> +
> + /**
> + * Height in pixels
> + */
> + int height;
> +
> + /**
> + * Horizonal coordinate of focal point in pixels
> + */
> + double focal_point_x;
> +
> + /**
> + * Vertical coordinate of focal point in pixels
> + */
> + double focal_point_y;
> +} Camera;
> +
> +/**
> + * Motion stabilization algorithm, mirroring those available in
> libdewobble.
> + */
> +typedef enum StabilizationAlgorithm {
> +
> + /**
> + * Do not apply stabilization
> + */
> + STABILIZATION_ALGORITHM_ORIGINAL,
> +
> + /**
> + * Keep the camera orientation fixed at its orientation in the first
> frame
> + */
> + STABILIZATION_ALGORITHM_FIXED,
> +
> + /**
> + * Smooth camera orientation with a Savitsky-Golay filter
> + */
> + STABILIZATION_ALGORITHM_SMOOTH,
> +
> + /**
> + * Number of stabilization algorithms
> + */
> + NB_STABILIZATION_ALGORITHMS,
> +
> +} StabilizationAlgorithm;
> +
>
Huh? Why this and bellow similar stuff are not part of library?
+/**
> + * libdewobble_opencl filter context
> + */
> +typedef struct LibDewobbleOpenCLContext {
> +
> + /**
> + * Generic OpenCL filter context
> + */
> + OpenCLFilterContext ocf;
> +
> + /**
> + * OpenCL command queue
> + */
> + cl_command_queue command_queue;
> +
> + /**
> + * Input camera (projection, focal length, etc)
> + */
> + Camera input_camera;
> +
> + /**
> + * Output camera (projection, focal length, etc)
> + */
> + Camera output_camera;
> +
> + /**
> + * Stabilization algorithm applied by the filter
> + * (@ref StabilizationAlgorithm)
> + */
> + int stabilization_algorithm;
> +
> + /**
> + * The number of frames to look ahead and behind for the purpose of
> + * stabilizing each frame
> + */
> + int stabilization_radius;
> +
> + /**
> + * The number of frames to look ahead for the purpose of interpolating
> + * frame rotation for frames where detection fails
> + */
> + int stabilization_horizon;
> +
> + /**
> + * The algorithm to interpolate the value between source image pixels
> + * (e.g.\ `DEWOBBLE_INTERPOLATION_LINEAR`)
> + */
> + int interpolation_algorithm;
> +
> + /**
> + * The algorithm used to fill in unmapped areas of the output (e.g.\
> + * `DEWOBBLE_BORDER_CONSTANT`)
> + */
> + int border_type;
> +
> + /**
> + * The color used to fill unmapped areas of the output when
> + * @ref border_type is `DEWOBBLE_BORDER_CONSTANT`
> + */
> + double border_color[4];
> +
> + /**
> + * Whether to include debugging information in the output
> + */
> + int debug;
> +
> + /**
> + * Whether the filter has been initialized
> + */
> + int initialized;
> +
> + /**
> + * The status of the input link
> + */
> + int input_status;
> +
> + /**
> + * The time that the input status was reached
> + */
> + int64_t input_status_pts;
> +
> + /**
> + * Number of frame jobs currently in progress (read from inlink but
> not
> + * yet sent to outlink)
> + */
> + int nb_frames_in_progress;
> +
> + /**
> + * Number of frames consumed so far
> + */
> + long nb_frames_consumed;
> +
> + /**
> + * The instance of libdewobble's filter
> + */
> + DewobbleFilter dewobble_filter;
> +
> +} LibDewobbleOpenCLContext;
> +
> +/**
> + * Convert degrees to radians.
> + * @param degrees the number of degrees
> + * @return the equivalent number of radians
> + */
> +static double degrees_to_radians(double degrees)
> +{
> + return degrees * M_PI / 180;
> +}
> +
> +/**
> + * Initialize the libdewobble filter instance.
> + * @param avctx the filter context
> + * @return 0 on success, otherwise a negative error code
> + */
> +static int init_libdewobble_filter(AVFilterContext *avctx)
> +{
> + LibDewobbleOpenCLContext *ctx = avctx->priv;
> + DewobbleStabilizer stabilizer = NULL;
> + DewobbleCamera input_camera = NULL, output_camera = NULL;
> + DewobbleFilterConfig config = NULL;
> +
> + input_camera = dewobble_camera_create(
> + ctx->input_camera.model,
> degrees_to_radians(ctx->input_camera.diagonal_fov),
> + ctx->input_camera.width, ctx->input_camera.height,
> + ctx->input_camera.focal_point_x, ctx->input_camera.focal_point_y);
> +
> + if (input_camera == NULL)
> + goto fail;
> +
> + output_camera = dewobble_camera_create(
> + ctx->output_camera.model,
> + degrees_to_radians(ctx->output_camera.diagonal_fov),
> + ctx->output_camera.width, ctx->output_camera.height,
> + ctx->output_camera.focal_point_x,
> ctx->output_camera.focal_point_y);
> +
> + if (output_camera == NULL)
> + goto fail;
> +
> + switch (ctx->stabilization_algorithm) {
> + case STABILIZATION_ALGORITHM_ORIGINAL:
> + stabilizer = dewobble_stabilizer_create_none();
> + break;
> + case STABILIZATION_ALGORITHM_FIXED:
> + stabilizer = dewobble_stabilizer_create_fixed(input_camera,
> +
> ctx->stabilization_horizon);
> +
> + break;
> + case STABILIZATION_ALGORITHM_SMOOTH:
> + stabilizer = dewobble_stabilizer_create_savitzky_golay(
> + input_camera, ctx->stabilization_radius,
> ctx->stabilization_horizon);
> +
> + break;
> + }
> +
> + if (stabilizer == NULL)
> + goto fail;
> +
> + config = dewobble_filter_config_create(input_camera, output_camera,
> stabilizer);
> +
> + dewobble_filter_config_set_opencl_context(config,
> ctx->ocf.hwctx->context);
> + dewobble_filter_config_set_opencl_device(config,
> ctx->ocf.hwctx->device_id);
> + dewobble_filter_config_set_interpolation(config,
> ctx->interpolation_algorithm);
> + dewobble_filter_config_set_border_type(config, ctx->border_type);
> + dewobble_filter_config_set_border_color(config, ctx->border_color);
> + dewobble_filter_config_set_debug(config, ctx->debug);
> +
> + ctx->dewobble_filter = dewobble_filter_create_threaded(config);
> +
> + dewobble_filter_config_destroy(&config);
> +
> + if (ctx->dewobble_filter == NULL)
> + goto fail;
> +
> + dewobble_stabilizer_destroy(&stabilizer);
> +
> + return 0;
> +
> +fail:
> + dewobble_stabilizer_destroy(&stabilizer);
> + dewobble_camera_destroy(&input_camera);
> + dewobble_camera_destroy(&output_camera);
> +
> + return AVERROR(ENOMEM);
> +}
> +
> +/**
> + * Initialize the filter based on the options
> + * @param avctx the filter context
> + * @return 0 on success, otherwise a negative error code
> + */
> +static int libdewobble_opencl_init(AVFilterContext *avctx)
> +{
> + LibDewobbleOpenCLContext *ctx = avctx->priv;
> +
> + av_log(avctx, AV_LOG_VERBOSE, "Init\n");
> +
> + if (ctx->input_camera.model == DEWOBBLE_NB_PROJECTIONS
> + || ctx->output_camera.model == DEWOBBLE_NB_PROJECTIONS) {
> +
> + av_log(avctx, AV_LOG_ERROR, "both in_p and out_p must be set\n");
> + return AVERROR(EINVAL);
> + }
> +
> + if (ctx->input_camera.diagonal_fov == 0 ||
> ctx->output_camera.diagonal_fov == 0) {
> + av_log(avctx, AV_LOG_ERROR, "both in_dfov and out_dfov must be
> set\n");
> + return AVERROR(EINVAL);
> + }
> +
> + if (ctx->stabilization_algorithm == STABILIZATION_ALGORITHM_ORIGINAL)
> + ctx->stabilization_horizon = 0;
> +
> + return ff_opencl_filter_init(avctx);
> +}
> +
> +/**
> + * Clean up the filter on destruction.
> + * @param avctx the filter context
> + */
> +static void libdewobble_opencl_uninit(AVFilterContext *avctx)
> +{
> + cl_int cle;
> + LibDewobbleOpenCLContext *ctx = avctx->priv;
> +
> + av_log(avctx, AV_LOG_VERBOSE, "Uninit\n");
> +
> + if (ctx->command_queue) {
> + cle = clReleaseCommandQueue(ctx->command_queue);
> +
> + if (cle != CL_SUCCESS)
> + av_log(avctx, AV_LOG_ERROR,
> + "Failed to release command queue: %d.\n", cle);
> + }
> +
> + dewobble_filter_destroy(&ctx->dewobble_filter);
> + ff_opencl_filter_uninit(avctx);
> +}
> +
> +/**
> + * Perform further initialization of the filter when the first input
> frame is
> + * available.
> + * @param avctx the filter context
> + * @param first_frame the first input frame
> + * @return 0 on success, otherwise a negative error code
> + */
> +static int libdewobble_opencl_frames_init(AVFilterContext *avctx, AVFrame
> *first_frame)
> +{
> + LibDewobbleOpenCLContext *ctx = avctx->priv;
> + AVFilterLink *inlink = avctx->inputs[0];
> + cl_int cle;
> + int err;
> +
> + if (first_frame->crop_top % 2 == 1 || first_frame->crop_bottom % 2 ==
> 1
> + || first_frame->crop_left % 2 == 1 || first_frame->crop_right % 2
> == 1) {
> +
> + av_log(avctx, AV_LOG_ERROR,
> + "Cropping by an odd number of pixels is not supported!\n");
> + return AVERROR(EINVAL);
> + }
> +
> + if ((first_frame->crop_top || first_frame->crop_bottom)
> + && (ctx->output_camera.height == 0
> + || ctx->output_camera.focal_point_y == DBL_MAX))
> + av_log(avctx, AV_LOG_WARNING,
> + "Input is vertically cropped, but output height or
> vertical "
> + "focal point is not set. The default values are based on
> the "
> + "uncropped input!\n");
> +
> + if ((first_frame->crop_left || first_frame->crop_right)
> + && (ctx->output_camera.width == 0
> + || ctx->output_camera.focal_point_x == DBL_MAX))
> + av_log(avctx, AV_LOG_WARNING,
> + "Input is horizontally cropped, but output width or
> horizontal "
> + "focal point is not set. The default values are based on
> the "
> + "uncropped input!\n");
> +
> + ctx->input_camera.width
> + = inlink->w - first_frame->crop_left - first_frame->crop_right;
> + ctx->input_camera.height
> + = inlink->h - first_frame->crop_top - first_frame->crop_bottom;
> +
> + /* Output camera width must match the filter output */
> + ctx->output_camera.width = ctx->ocf.output_width;
> + ctx->output_camera.height = ctx->ocf.output_height;
> +
> + /* Focal points default to the image center (disregarding cropping) */
> + if (ctx->input_camera.focal_point_x == DBL_MAX)
> + ctx->input_camera.focal_point_x
> + = (inlink->w - 1) / 2.0 - first_frame->crop_left;
> +
> + if (ctx->input_camera.focal_point_y == DBL_MAX)
> + ctx->input_camera.focal_point_y
> + = (inlink->h - 1) / 2.0 - first_frame->crop_top;
> +
> + if (ctx->output_camera.focal_point_x == DBL_MAX)
> + ctx->output_camera.focal_point_x = (ctx->output_camera.width - 1)
> / 2.0;
> +
> + if (ctx->output_camera.focal_point_y == DBL_MAX)
> + ctx->output_camera.focal_point_y = (ctx->output_camera.height -
> 1) / 2.0;
> +
> + ctx->command_queue = clCreateCommandQueue(ctx->ocf.hwctx->context,
> + ctx->ocf.hwctx->device_id,
> 0, &cle);
> +
> + if (cle) {
> + av_log(avctx, AV_LOG_ERROR,
> + "Failed to create OpenCL command queue %d.\n", cle);
> + return AVERROR(EIO);
> + }
> +
> + err = init_libdewobble_filter(avctx);
> + if (err) {
> + av_log(avctx, AV_LOG_ERROR,
> + "Failed to initialise libdewobble filter %d.\n", err);
> + return AVERROR(EIO);
> + }
> +
> + ctx->initialized = 1;
> +
> + return 0;
> +}
> +
> +/**
> + * Perform initialization based on the input filter link.
> + * @param inlink the input filter link
> + * @return 0 on success, otherwise a negative error code
> + */
> +static int libdewobble_opencl_config_input(AVFilterLink *inlink)
> +{
> + AVFilterContext *avctx = inlink->dst;
> + LibDewobbleOpenCLContext *ctx = avctx->priv;
> + int ret;
> +
> + ret = ff_opencl_filter_config_input(inlink);
> +
> + if (ret < 0)
> + return ret;
> +
> + if (ctx->ocf.output_format != AV_PIX_FMT_NV12) {
> + av_log(avctx, AV_LOG_ERROR, "Only NV12 input is supported!\n");
> + return AVERROR(ENOSYS);
> + }
> +
> + if (inlink->w % 2 == 1 || inlink->h % 2 == 1) {
> + av_log(avctx, AV_LOG_ERROR, "Input with odd dimensions is not
> supported!\n");
> + return AVERROR(EINVAL);
> + }
> +
> + if (ctx->output_camera.width % 2 == 1 || ctx->output_camera.height %
> 2 == 1) {
> + av_log(avctx, AV_LOG_ERROR, "Output camera must have even
> dimensions!\n");
> + return AVERROR(EINVAL);
> + }
> +
> + /* Output dimensions default to the input dimensions (disregarding
> cropping) */
> + ctx->ocf.output_width
> + = ctx->output_camera.width ? ctx->output_camera.width : inlink->w;
> +
> + ctx->ocf.output_height
> + = ctx->output_camera.height ? ctx->output_camera.height :
> inlink->h;
> +
> + return 0;
> +}
> +
> +/**
> + * Copy the contents of an input frame to an OpenCL buffer.
> + * @param avctx the filter context
> + * @param context the OpenCL context to use
> + * @param command_queue the OpenCL command queue to use
> + * @param frame the input @ref AVFrame
> + * @param input_buffer the OpenCL buffer to copy the frame into
> + * @return 0 on success, otherwise a negative error code
> + */
> +static cl_int copy_frame_to_buffer(AVFilterContext *avctx, cl_context
> context,
> + cl_command_queue command_queue,
> + AVFrame *frame, cl_mem input_buffer)
> +{
> + int err;
> + LibDewobbleOpenCLContext *ctx = avctx->priv;
> + cl_mem luma = (cl_mem)frame->data[0];
> + cl_mem chroma = (cl_mem)frame->data[1];
> + cl_int cle = 0;
> + size_t src_luma_origin[3] = { frame->crop_left, frame->crop_top, 0 };
> +
> + size_t src_chroma_origin[3] = {
> + frame->crop_left / 2,
> + frame->crop_top / 2,
> + 0,
> + };
> +
> + size_t luma_region[3] = {
> + ctx->input_camera.width,
> + ctx->input_camera.height,
> + 1,
> + };
> +
> + size_t chroma_region[3] = {
> + ctx->input_camera.width / 2,
> + ctx->input_camera.height / 2,
> + 1,
> + };
> +
> + cl_event copy_finished[2];
> +
> + cle = clEnqueueCopyImageToBuffer(command_queue, luma, input_buffer,
> + src_luma_origin, luma_region, 0, 0,
> NULL,
> + ©_finished[0]);
> +
> + CL_FAIL_ON_ERROR(AVERROR(EINVAL),
> + "Failed to enqueue copy luma image to buffer: %d\n",
> cle);
> +
> + cle = clEnqueueCopyImageToBuffer(
> + command_queue, chroma, input_buffer, src_chroma_origin,
> chroma_region,
> + ctx->input_camera.width * ctx->input_camera.height * 1, 0, NULL,
> + ©_finished[1]);
> +
> + CL_FAIL_ON_ERROR(AVERROR(EINVAL),
> + "Failed to enqueue copy chroma image to buffer:
> %d\n", cle);
> +
> + cle = clWaitForEvents(2, copy_finished);
> +
> + CL_FAIL_ON_ERROR(AVERROR(EINVAL), "Failed to copy images to buffer:
> %d\n", cle);
> +
> + return 0;
> +
> +fail:
> + return err;
> +}
> +
> +/**
> + * Copy the contents of an OpenCL buffer to an output frame.
> + * @param avctx the filter context
> + * @param buffer the OpenCL buffer
> + * @param output_frame the output frame
> + * @return 0 on success, otherwise a negative error code
> + */
> +static int copy_buffer_to_frame(AVFilterContext *avctx, cl_mem buffer,
> + AVFrame *output_frame)
> +{
> + int err;
> + LibDewobbleOpenCLContext *ctx = avctx->priv;
> + cl_mem luma = (cl_mem)output_frame->data[0];
> + cl_mem chroma = (cl_mem)output_frame->data[1];
> + cl_int cle = 0;
> + size_t dst_origin[3] = { 0, 0, 0 };
> + size_t luma_region[3] = { output_frame->width, output_frame->height,
> 1 };
> +
> + size_t chroma_region[3] = {
> + output_frame->width / 2,
> + output_frame->height / 2,
> + 1,
> + };
> +
> + cl_event copy_finished[2];
> +
> + cle = clEnqueueCopyBufferToImage(ctx->command_queue, buffer, luma, 0,
> dst_origin,
> + luma_region, 0, NULL,
> ©_finished[0]);
> +
> + CL_FAIL_ON_ERROR(AVERROR(EINVAL),
> + "Failed to enqueue copy buffer to luma image: %d\n",
> cle);
> +
> + cle = clEnqueueCopyBufferToImage(ctx->command_queue, buffer, chroma,
> + output_frame->width *
> output_frame->height * 1,
> + dst_origin, chroma_region, 0, NULL,
> + ©_finished[1]);
> +
> + CL_FAIL_ON_ERROR(AVERROR(EINVAL),
> + "Failed to enqueue copy buffer to luma image: %d\n",
> cle);
> +
> + cle = clWaitForEvents(2, copy_finished);
> + CL_FAIL_ON_ERROR(AVERROR(EINVAL), "Failed to copy buffer to images:
> %d\n", cle);
> +
> + return 0;
> +
> +fail:
> + return err;
> +}
> +
> +/**
> + * Consume an input frame and push it to the libdewobble filter.
> + * @param avctx the filter context
> + * @param input_frame the input frame
> + * @return 0 on success, otherwise a negative error code
> + */
> +static int consume_input_frame(AVFilterContext *avctx, AVFrame
> *input_frame)
> +{
> + LibDewobbleOpenCLContext *ctx = avctx->priv;
> + cl_mem input_buffer;
> + int err = 0;
> + cl_int cle;
> +
> + if (!input_frame->hw_frames_ctx)
> + return AVERROR(EINVAL);
> +
> + if (!ctx->initialized) {
> + av_log(avctx, AV_LOG_VERBOSE, "Initializing\n");
> + err = libdewobble_opencl_frames_init(avctx, input_frame);
> +
> + if (err < 0)
> + return err;
> + }
> +
> + input_buffer
> + = dewobble_filter_get_input_frame_buffer(ctx->dewobble_filter,
> &cle);
> +
> + CL_FAIL_ON_ERROR(AVERROR(ENOMEM), "Failed to create buffer: %d\n",
> cle);
> +
> + err = copy_frame_to_buffer(avctx, ctx->ocf.hwctx->context,
> + ctx->command_queue, input_frame,
> input_buffer);
> +
> + if (err)
> + goto fail;
> +
> + /* Free original input frame buffers */
> + for (int i = 0; input_frame->buf[i] != NULL; i++)
> + av_buffer_unref(&input_frame->buf[i]);
> +
> + dewobble_filter_push_frame(ctx->dewobble_filter, input_buffer,
> + (void **)input_frame);
> +
> + ctx->nb_frames_in_progress += 1;
> + ctx->nb_frames_consumed += 1;
> +
> + return 0;
> +
> +fail:
> + return err;
> +}
> +
> +/**
> + * Create and send on an output frame using an output buffer pulled from
> the
> + * libdewobble filter.
> + * @param avctx the filter context
> + * @return 0 on success, otherwise a negative error code.
> + */
> +static int send_output_frame(AVFilterContext *avctx)
> +{
> + AVFilterLink *inlink = avctx->inputs[0];
> + AVFilterLink *outlink = avctx->outputs[0];
> + LibDewobbleOpenCLContext *ctx = avctx->priv;
> + AVFrame *input_frame;
> + AVFrame *output_frame = NULL;
> + cl_mem output_buffer = NULL, input_buffer;
> + int err;
> +
> + dewobble_filter_pull_frame(ctx->dewobble_filter, &output_buffer,
> + &input_buffer, (void **)&input_frame);
> +
> + dewobble_filter_release_input_frame_buffer(ctx->dewobble_filter,
> &input_buffer);
> +
> + output_frame = ff_get_video_buffer(outlink, outlink->w, outlink->h);
> + if (output_frame == NULL) {
> + err = AVERROR(ENOMEM);
> + goto fail;
> + }
> +
> + err = av_frame_copy_props(output_frame, input_frame);
> + if (err)
> + goto fail;
> +
> + output_frame->crop_top = 0;
> + output_frame->crop_bottom = 0;
> + output_frame->crop_left = 0;
> + output_frame->crop_right = 0;
> +
> + err = copy_buffer_to_frame(avctx, output_buffer, output_frame);
> + if (err)
> + goto fail;
> +
> + dewobble_filter_release_output_frame_buffer(ctx->dewobble_filter,
> &output_buffer);
> +
> + av_log(avctx, AV_LOG_VERBOSE, "Sending output frame %ld (%d in
> progress)\n",
> + ctx->nb_frames_consumed - ctx->nb_frames_in_progress,
> + ctx->nb_frames_in_progress);
> +
> + ctx->nb_frames_in_progress -= 1;
> +
> + err = ff_filter_frame(outlink, output_frame);
> + if (err < 0)
> + goto fail;
> +
> + if (!dewobble_filter_frame_ready(ctx->dewobble_filter))
> + ff_inlink_request_frame(inlink);
> +
> + if (ctx->input_status && ctx->nb_frames_in_progress == 0) {
> + av_log(avctx, AV_LOG_VERBOSE, "Output reached EOF\n");
> + ff_outlink_set_status(outlink, ctx->input_status,
> ctx->input_status_pts);
> + }
> +
> + av_frame_free(&input_frame);
> +
> + return 0;
> +
> +fail:
> + av_frame_free(&input_frame);
> + av_log(avctx, AV_LOG_ERROR, "Failed to send output frame: %d\n", err);
> + av_frame_free(&output_frame);
> +
> + return err;
> +}
> +
> +/**
> + * Attempt to consume an input frame, and push it to the libdewobble
> filter
> + * if one is available.
> + * @param avctx the filter context
> + * @return 0 on success, otherwise a negative error code
> + */
> +static int try_consume_input_frame(AVFilterContext *avctx)
> +{
> + AVFilterLink *inlink = avctx->inputs[0];
> + LibDewobbleOpenCLContext *ctx = avctx->priv;
> + int err = 0;
> + AVFrame *input_frame;
> +
> + /* If necessary, attempt to consume a frame from the input */
> + if (!ctx->initialized ||
> !dewobble_filter_frame_ready(ctx->dewobble_filter)) {
> + err = ff_inlink_consume_frame(inlink, &input_frame);
> + if (err < 0) {
> + av_log(avctx, AV_LOG_ERROR, "Failed to read input frame\n");
> +
> + return err;
> + } else if (err > 0) {
> + av_log(avctx, AV_LOG_VERBOSE,
> + "Consuming input frame %ld (%d in progress)\n",
> + ctx->nb_frames_consumed, ctx->nb_frames_in_progress);
> +
> + err = consume_input_frame(avctx, input_frame);
> + if (err) {
> + av_log(avctx, AV_LOG_ERROR,
> + "Failed to consume input frame: %d\n", err);
> +
> + return err;
> + }
> + }
> + }
> +
> + return err;
> +}
> +
> +/**
> + * Read the input status and update the filter state and output status as
> + * appropriate.
> + * @param avctx the filter context
> + */
> +static void check_input_status(AVFilterContext *avctx)
> +{
> + AVFilterLink *inlink = avctx->inputs[0];
> + AVFilterLink *outlink = avctx->outputs[0];
> + LibDewobbleOpenCLContext *ctx = avctx->priv;
> +
> + /* Check for end of input */
> + if (!ctx->input_status
> + && ff_inlink_acknowledge_status(inlink, &ctx->input_status,
> + &ctx->input_status_pts)) {
> +
> + if (ctx->input_status == AVERROR_EOF) {
> + av_log(avctx, AV_LOG_VERBOSE, "Reached input EOF\n");
> +
> + dewobble_filter_end_input(ctx->dewobble_filter);
> + } else
> + av_log(avctx, AV_LOG_ERROR, "Input status: %d\n",
> ctx->input_status);
> +
> + if (ctx->nb_frames_in_progress == 0) {
> + av_log(avctx, AV_LOG_VERBOSE, "Sending output EOF\n");
> +
> + ff_outlink_set_status(outlink, ctx->input_status,
> ctx->input_status_pts);
> + }
> + }
> +}
> +
> +/**
> + * Perform some work to advance the filtering process
> + * @param avctx the filter context
> + * @return 0 if progress was made, otherwise a negative error code
> + */
> +static int activate(AVFilterContext *avctx)
> +{
> + LibDewobbleOpenCLContext *ctx = avctx->priv;
> + AVFilterLink *inlink = avctx->inputs[0];
> + AVFilterLink *outlink = avctx->outputs[0];
> + int err = 0;
> +
> + /* Forward any output status to input */
> + err = ff_outlink_get_status(outlink);
> + if (err) {
> + av_log(avctx, AV_LOG_VERBOSE, "forwarding status to inlink:
> %d\n", err);
> + ff_inlink_set_status(inlink, err);
> + return 0;
> + }
> +
> + /* Consume an input frame if possible */
> + err = try_consume_input_frame(avctx);
> + if (err) {
> + av_log(avctx, AV_LOG_ERROR, "try_consume_input_frame failed:
> %d\n", err);
> + return 0;
> + }
> +
> + /* Check input status, including detecting EOF */
> + check_input_status(avctx);
> +
> + /* If possible, send an output frame */
> + if (dewobble_filter_frame_ready(ctx->dewobble_filter)) {
> + err = send_output_frame(avctx);
> + if (err < 0) {
> + av_log(avctx, AV_LOG_ERROR, "send_output_frame failed: %d\n",
> err);
> + goto fail;
> + }
> + }
> +
> + /* Schedule the next activation */
> + if (ff_inlink_check_available_frame(inlink))
> + /* Immediately, if input frames are still queued */
> + ff_filter_set_ready(avctx, 1);
> + else if (dewobble_filter_frame_ready(ctx->dewobble_filter))
> + /* Immediately, if output frames are ready */
> + ff_filter_set_ready(avctx, 1);
> + else
> + /* Otherwise when more input frames are ready */
> + ff_inlink_request_frame(inlink);
> +
> + return FFERROR_NOT_READY;
> +
> +fail:
> + ff_outlink_set_status(outlink, AVERROR_UNKNOWN, 0);
> + return err;
> +}
> +
> +/**
> + * Get the offset of a member in @ref LibDewobbleOpenCLContext
> + */
> +#define OFFSET(x) offsetof(LibDewobbleOpenCLContext, x)
> +
> +/**
> + * Get the offset of a member in @ref Camera
> + */
> +#define OFFSET_CAMERA(x) offsetof(Camera, x)
> +
> +#define FLAGS (AV_OPT_FLAG_FILTERING_PARAM | AV_OPT_FLAG_VIDEO_PARAM)
> +
> +static const AVOption libdewobble_opencl_options[] = {
> + /* Input camera options */
> + {
> + "in_p",
> + "input camera projection model",
> + OFFSET(input_camera) + OFFSET_CAMERA(model),
> + AV_OPT_TYPE_INT,
> + { .i64 = DEWOBBLE_PROJECTION_EQUIDISTANT_FISHEYE },
> + 0,
> + DEWOBBLE_NB_PROJECTIONS - 1,
> + FLAGS,
> + "model",
> + },
> + {
> + "in_dfov",
> + "input camera diagonal field of view in degrees",
> + OFFSET(input_camera) + OFFSET_CAMERA(diagonal_fov),
> + AV_OPT_TYPE_DOUBLE,
> + { .dbl = 0 },
> + 0,
> + DBL_MAX,
> + .flags = FLAGS,
> + },
> + {
> + "in_fx",
> + "horizontal coordinate of focal point in input camera (default: "
> + "center)",
> + OFFSET(input_camera) + OFFSET_CAMERA(focal_point_x),
> + AV_OPT_TYPE_DOUBLE,
> + { .dbl = DBL_MAX },
> + -DBL_MAX,
> + DBL_MAX,
> + .flags = FLAGS,
> + },
> + {
> + "in_fy",
> + "vertical coordinate of focal point in input camera (default:
> center)",
> + OFFSET(input_camera) + OFFSET_CAMERA(focal_point_y),
> + AV_OPT_TYPE_DOUBLE,
> + { .dbl = DBL_MAX },
> + -DBL_MAX,
> + DBL_MAX,
> + .flags = FLAGS,
> + },
> +
> + /* Output camera options */
> + {
> + "out_p",
> + "output camera projection model",
> + OFFSET(output_camera) + OFFSET_CAMERA(model),
> + AV_OPT_TYPE_INT,
> + { .i64 = DEWOBBLE_PROJECTION_RECTILINEAR },
> + 0,
> + DEWOBBLE_NB_PROJECTIONS - 1,
> + FLAGS,
> + "model",
> + },
> + {
> + "out_dfov",
> + "output camera diagonal field of view in degrees",
> + OFFSET(output_camera) + OFFSET_CAMERA(diagonal_fov),
> + AV_OPT_TYPE_DOUBLE,
> + { .dbl = 0 },
> + 0,
> + DBL_MAX,
> + .flags = FLAGS,
> + },
> + {
> + "out_w",
> + "output camera width in pixels (default: same as input)",
> + OFFSET(output_camera) + OFFSET_CAMERA(width),
> + AV_OPT_TYPE_INT,
> + { .i64 = 0 },
> + 0,
> + SHRT_MAX,
> + .flags = FLAGS,
> + },
> + {
> + "out_h",
> + "output camera height in pixels (default: same as input)",
> + OFFSET(output_camera) + OFFSET_CAMERA(height),
> + AV_OPT_TYPE_INT,
> + { .i64 = 0 },
> + 0,
> + SHRT_MAX,
> + .flags = FLAGS,
> + },
> + {
> + "out_fx",
> + "horizontal coordinate of focal point in output camera "
> + "(default: center)",
> + OFFSET(output_camera) + OFFSET_CAMERA(focal_point_x),
> + AV_OPT_TYPE_DOUBLE,
> + { .dbl = DBL_MAX },
> + -DBL_MAX,
> + DBL_MAX,
> + .flags = FLAGS,
> + },
> + {
> + "out_fy",
> + "vertical coordinate of focal point in output camera "
> + "(default: center)",
> + OFFSET(output_camera) + OFFSET_CAMERA(focal_point_y),
> + AV_OPT_TYPE_DOUBLE,
> + { .dbl = DBL_MAX },
> + -DBL_MAX,
> + DBL_MAX,
> + .flags = FLAGS,
> + },
> +
> + /* Stabilization options */
> + {
> + "stab",
> + "camera orientation stabilization algorithm",
> + OFFSET(stabilization_algorithm),
> + AV_OPT_TYPE_INT,
> + { .i64 = STABILIZATION_ALGORITHM_SMOOTH },
> + 0,
> + NB_STABILIZATION_ALGORITHMS - 1,
> + FLAGS,
> + "stab",
> + },
> + {
> + "stab_r",
> + "for Savitzky-Golay smoothing: the number of frames "
> + "to look ahead and behind",
> + OFFSET(stabilization_radius),
> + AV_OPT_TYPE_INT,
> + { .i64 = 15 },
> + 1,
> + INT_MAX,
> + FLAGS,
> + },
> + {
> + "stab_h",
> + "for stabilization: the number of frames to look "
> + "ahead to interpolate rotation in frames where it cannot be
> detected",
> + OFFSET(stabilization_horizon),
> + AV_OPT_TYPE_INT,
> + { .i64 = 30 },
> + 0,
> + INT_MAX,
> + FLAGS,
> + },
> +
> + /* General options */
> + {
> + "interp",
> + "interpolation algorithm",
> + OFFSET(interpolation_algorithm),
> + AV_OPT_TYPE_INT,
> + { .i64 = DEWOBBLE_INTERPOLATION_LINEAR },
> + 0,
> + DEWOBBLE_NB_INTERPOLATIONS - 1,
> + FLAGS,
> + "interpolation",
> + },
> + {
> + "border",
> + "border fill mode",
> + OFFSET(border_type),
> + AV_OPT_TYPE_INT,
> + { .i64 = DEWOBBLE_BORDER_CONSTANT },
> + 0,
> + DEWOBBLE_NB_BORDER_TYPES - 1,
> + FLAGS,
> + "border_type",
> + },
> + {
> + "border_r",
> + "border fill color (red component)",
> + OFFSET(border_color) + sizeof(double) * 2,
> + AV_OPT_TYPE_DOUBLE,
> + { .i64 = 0 },
> + 0,
> + 255,
> + FLAGS,
> + },
> + {
> + "border_g",
> + "border fill color (green component)",
> + OFFSET(border_color) + sizeof(double) * 1,
> + AV_OPT_TYPE_DOUBLE,
> + { .i64 = 0 },
> + 0,
> + 255,
> + FLAGS,
> + },
> + {
> + "border_b",
> + "border fill color (blue component)",
> + OFFSET(border_color) + sizeof(double) * 0,
> + AV_OPT_TYPE_DOUBLE,
> + { .i64 = 0 },
> + 0,
> + 255,
> + FLAGS,
> + },
> + {
> + "debug",
> + "whether to include debugging information in the output",
> + OFFSET(debug),
> + AV_OPT_TYPE_BOOL,
> + { .i64 = 0 },
> + 0,
> + 1,
> + FLAGS,
> + },
> +
> + /* Camera models */
> + {
> + "rect",
> + "rectilinear projection",
> + 0,
> + AV_OPT_TYPE_CONST,
> + { .i64 = DEWOBBLE_PROJECTION_RECTILINEAR },
> + INT_MIN,
> + INT_MAX,
> + FLAGS,
> + "model",
> + },
> + {
> + "fish",
> + "equidistant fisheye projection",
> + 0,
> + AV_OPT_TYPE_CONST,
> + { .i64 = DEWOBBLE_PROJECTION_EQUIDISTANT_FISHEYE },
> + INT_MIN,
> + INT_MAX,
> + FLAGS,
> + "model",
> + },
> +
> + /* Stabilization algorithms */
> + {
> + "fixed",
> + "fix the camera orientation after the first frame",
> + 0,
> + AV_OPT_TYPE_CONST,
> + { .i64 = STABILIZATION_ALGORITHM_FIXED },
> + INT_MIN,
> + INT_MAX,
> + FLAGS,
> + "stab",
> + },
> + {
> + "none",
> + "do not apply stabilization",
> + 0,
> + AV_OPT_TYPE_CONST,
> + { .i64 = STABILIZATION_ALGORITHM_ORIGINAL },
> + INT_MIN,
> + INT_MAX,
> + FLAGS,
> + "stab",
> + },
> + {
> + "sg",
> + "smooth the camera orientation using a Savitzky-Golay filter",
> + 0,
> + AV_OPT_TYPE_CONST,
> + { .i64 = STABILIZATION_ALGORITHM_SMOOTH },
> + INT_MIN,
> + INT_MAX,
> + FLAGS,
> + "stab",
> + },
> +
> + /* Interpolation algorithms */
> + {
> + "nearest",
> + "nearest neighbour interpolation (fast)",
> + 0,
> + AV_OPT_TYPE_CONST,
> + { .i64 = DEWOBBLE_INTERPOLATION_NEAREST },
> + INT_MIN,
> + INT_MAX,
> + FLAGS,
> + "interpolation",
> + },
> + {
> + "linear",
> + "bilinear interpolation (fast)",
> + 0,
> + AV_OPT_TYPE_CONST,
> + { .i64 = DEWOBBLE_INTERPOLATION_LINEAR },
> + INT_MIN,
> + INT_MAX,
> + FLAGS,
> + "interpolation",
> + },
> + {
> + "cubic",
> + "bicubic interpolation (medium)",
> + 0,
> + AV_OPT_TYPE_CONST,
> + { .i64 = DEWOBBLE_INTERPOLATION_CUBIC },
> + INT_MIN,
> + INT_MAX,
> + FLAGS,
> + "interpolation",
> + },
> + {
> + "lanczos",
> + "Lanczos4, in an 8x8 neighbourhood (slow)",
> + 0,
> + AV_OPT_TYPE_CONST,
> + { .i64 = DEWOBBLE_INTERPOLATION_LANCZOS4 },
> + INT_MIN,
> + INT_MAX,
> + FLAGS,
> + "interpolation",
> + },
> +
> + /* Border fill algorithms */
> + {
> + "constant",
> + "constant color (default black)",
> + 0,
> + AV_OPT_TYPE_CONST,
> + { .i64 = DEWOBBLE_BORDER_CONSTANT },
> + INT_MIN,
> + INT_MAX,
> + FLAGS,
> + "border_type",
> + },
> + {
> + "reflect",
> + "reflection of the input about the edge",
> + 0,
> + AV_OPT_TYPE_CONST,
> + { .i64 = DEWOBBLE_BORDER_REFLECT },
> + INT_MIN,
> + INT_MAX,
> + FLAGS,
> + "border_type",
> + },
> + {
> + "reflect101",
> + "reflection of the input about the middle of the pixel on the
> edge",
> + 0,
> + AV_OPT_TYPE_CONST,
> + { .i64 = DEWOBBLE_BORDER_REFLECT_101 },
> + INT_MIN,
> + INT_MAX,
> + FLAGS,
> + "border_type",
> + },
> + {
> + "replicate",
> + "replicate the pixel on the edge",
> + 0,
> + AV_OPT_TYPE_CONST,
> + { .i64 = DEWOBBLE_BORDER_REPLICATE },
> + INT_MIN,
> + INT_MAX,
> + FLAGS,
> + "border_type",
> + },
> + {
> + "wrap",
> + "wrap around to the opposite side of the source image",
> + 0,
> + AV_OPT_TYPE_CONST,
> + { .i64 = DEWOBBLE_BORDER_WRAP },
> + INT_MIN,
> + INT_MAX,
> + FLAGS,
> + "border_type",
> + },
> + { NULL },
> +};
> +
> +AVFILTER_DEFINE_CLASS(libdewobble_opencl);
> +
> +static const AVFilterPad inputs[] = {
> + {
> + .name = "default",
> + .type = AVMEDIA_TYPE_VIDEO,
> + .config_props = &libdewobble_opencl_config_input,
> + },
> +};
> +
> +static const AVFilterPad outputs[] = {
> + {
> + .name = "default",
> + .type = AVMEDIA_TYPE_VIDEO,
> + .config_props = &ff_opencl_filter_config_output,
> + },
> +};
> +
> +const AVFilter ff_vf_libdewobble_opencl = {
> + .name = "libdewobble_opencl",
> + .description = NULL_IF_CONFIG_SMALL(
> + "apply motion stabilization with awareness of camera projection "
>
Apply ....
> + "and/or change camera projection"),
> + .priv_size = sizeof(LibDewobbleOpenCLContext),
> + .priv_class = &libdewobble_opencl_class,
> + .init = &libdewobble_opencl_init,
> + .uninit = &libdewobble_opencl_uninit,
> + .query_formats = &ff_opencl_filter_query_formats,
> + FILTER_INPUTS(inputs),
> + FILTER_OUTPUTS(outputs),
> + .activate = activate,
> + .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE,
> +};
>
I really dislike GPL3 and use of opencv.
I would prefer pure C solution.
> --
> 2.33.0
>
>
More information about the ffmpeg-devel
mailing list