[FFmpeg-devel] [PATCH] Add alphaextract, alphamerge filters
Stefano Sabatini
stefasab at gmail.com
Wed Jul 11 15:43:09 CEST 2012
On date Tuesday 2012-07-10 22:41:06 -0700, Steven Robertson encoded:
> Hi all,
>
> I need to transmit a bunch of frames with a real-valued alpha channel
> (not just a static mask) over a relatively slow link, and since the
> recipient is on Windows, I'd like the relative simplicity of having
> FFmpeg be the only tool that he needs to recover the content. A bit of
> searching seems to indicate I'm not the first to want this, so I wrote
> up a simple pair of filters to make this possible. From the patch
> description:
>
> "These filters are designed for storing and transmitting video sequences
> with alpha using higher-efficiency codecs such as x264 which don't
> natively support an alpha channel. 'alphaextract' takes an input stream
> with an alpha channel and returns a video containing just the alpha
> component as a grayscale value; 'alphamerge' takes an RGB or YUV stream
> and adds an alpha channel recovered from a second grayscale stream."
>
> Happy to address any review comments.
>
> Thanks,
> Steve
> From c2709fecad9bbbad927f679fa2af6b17f9b2e4fa Mon Sep 17 00:00:00 2001
> From: Steven Robertson <steven at strobe.cc>
> Date: Tue, 10 Jul 2012 22:14:57 -0700
> Subject: [PATCH] Add alphaextract, alphamerge filters.
>
> These filters are designed for storing and transmitting video sequences
> with alpha using higher-efficiency codecs such as x264 which don't
> natively support an alpha channel. 'alphaextract' takes an input stream
> with an alpha channel and returns a video containing just the alpha
> component as a grayscale value; 'alphamerge' takes an RGB or YUV stream
> and adds an alpha channel recovered from a second grayscale stream.
>
> Signed-off-by: Steven Robertson <steven at strobe.cc>
> ---
> doc/filters.texi | 24 +++++
> libavfilter/Makefile | 2 +
> libavfilter/allfilters.c | 2 +
> libavfilter/vf_alphaextract.c | 108 +++++++++++++++++++++++
> libavfilter/vf_alphamerge.c | 186 +++++++++++++++++++++++++++++++++++++++
> tests/lavfi-regression.sh | 15 +++-
> tests/ref/lavfi/alphaextract_rgb | 1 +
> tests/ref/lavfi/alphaextract_yuv | 1 +
> tests/ref/lavfi/alphamerge_rgb | 1 +
> tests/ref/lavfi/alphamerge_yuv | 1 +
> 10 files changed, 338 insertions(+), 3 deletions(-)
> create mode 100644 libavfilter/vf_alphaextract.c
> create mode 100644 libavfilter/vf_alphamerge.c
> create mode 100644 tests/ref/lavfi/alphaextract_rgb
> create mode 100644 tests/ref/lavfi/alphaextract_yuv
> create mode 100644 tests/ref/lavfi/alphamerge_rgb
> create mode 100644 tests/ref/lavfi/alphamerge_yuv
>
> diff --git a/doc/filters.texi b/doc/filters.texi
> index 0d94eba..d9473b4 100644
> --- a/doc/filters.texi
> +++ b/doc/filters.texi
> @@ -1016,6 +1016,30 @@ build.
>
> Below is a description of the currently available video filters.
>
> + at section alphaextract
> +
> +Extract the alpha component from the input as a grayscale video. This
> +is especially useful with the @var{alphamerge} filter.
I suppose this could be made more general (compextract?), but I
don't object the inclusion of this.
> +
> + at section alphamerge
> +
> +Add or replace the alpha component of the primary input with the
> +grayscale value of a second input. This is intended for use with
> + at var{alphaextract} to allow the transmission or storage of frame
> +sequences that have alpha in a format that doesn't support an alpha
> +channel.
> +
> +For example, to reconstruct full frames from a normal YUV-encoded video
> +and a separate video created with @var{alphaextract}, you might use:
> + at example
> +movie=in_alpha.mkv [alpha]; [in][alpha] alphamerge [out]
> + at end example
> +
> +Since this filter is designed for reconstruction, it operates on frame
> +sequences without considering timestamps, and terminates when either
> +input reaches end of stream. If you're trying to apply an image as an
> +overlay to a video stream, consider the @var{overlay} filter instead.
> +
> @section ass
>
> Draw ASS (Advanced Substation Alpha) subtitles on top of input video
> diff --git a/libavfilter/Makefile b/libavfilter/Makefile
> index b094f59..a177752 100644
> --- a/libavfilter/Makefile
> +++ b/libavfilter/Makefile
> @@ -76,6 +76,8 @@ OBJS-$(CONFIG_ABUFFERSINK_FILTER) += sink_buffer.o
> OBJS-$(CONFIG_ANULLSINK_FILTER) += asink_anullsink.o
>
> OBJS-$(CONFIG_ASS_FILTER) += vf_ass.o
> +OBJS-$(CONFIG_ALPHAEXTRACT_FILTER) += vf_alphaextract.o
> +OBJS-$(CONFIG_ALPHAMERGE_FILTER) += vf_alphamerge.o
> OBJS-$(CONFIG_BBOX_FILTER) += bbox.o vf_bbox.o
> OBJS-$(CONFIG_BLACKDETECT_FILTER) += vf_blackdetect.o
> OBJS-$(CONFIG_BLACKFRAME_FILTER) += vf_blackframe.o
> diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
> index 706405e..aad4534 100644
> --- a/libavfilter/allfilters.c
> +++ b/libavfilter/allfilters.c
> @@ -64,6 +64,8 @@ void avfilter_register_all(void)
> REGISTER_FILTER (ABUFFERSINK, abuffersink, asink);
> REGISTER_FILTER (ANULLSINK, anullsink, asink);
>
> + REGISTER_FILTER (ALPHAEXTRACT, alphaextract, vf);
> + REGISTER_FILTER (ALPHAMERGE, alphamerge, vf);
> REGISTER_FILTER (ASS, ass, vf);
> REGISTER_FILTER (BBOX, bbox, vf);
> REGISTER_FILTER (BLACKDETECT, blackdetect, vf);
> diff --git a/libavfilter/vf_alphaextract.c b/libavfilter/vf_alphaextract.c
> new file mode 100644
> index 0000000..249c75d
> --- /dev/null
> +++ b/libavfilter/vf_alphaextract.c
> @@ -0,0 +1,108 @@
> +/*
> + * Copyright (c) 2012 Steven Robertson
> + *
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +
> +/**
> + * @file
> + * simple channel-swapping filter to get at the alpha component
> + */
> +
> +#include <string.h>
> +
> +#include "libavutil/avassert.h"
> +#include "libavutil/pixfmt.h"
> +#include "avfilter.h"
> +#include "drawutils.h"
> +#include "formats.h"
> +#include "video.h"
> +
> +enum { Y, U, V, A };
> +
> +typedef struct {
> + int is_packed_rgb;
> + uint8_t rgba_map[4];
> +} AlphaExtractContext;
> +
> +static int query_formats(AVFilterContext *ctx)
> +{
> + enum PixelFormat in_fmts[] = {
> + PIX_FMT_YUVA444P, PIX_FMT_YUVA422P, PIX_FMT_YUVA420P,
> + PIX_FMT_RGBA, PIX_FMT_BGRA, PIX_FMT_ARGB, PIX_FMT_ABGR,
> + PIX_FMT_NONE
> + };
> + enum PixelFormat out_fmts[] = { PIX_FMT_GRAY8, PIX_FMT_NONE };
> + ff_formats_ref(ff_make_format_list(in_fmts), &ctx->inputs[0]->out_formats);
> + ff_formats_ref(ff_make_format_list(out_fmts), &ctx->outputs[0]->in_formats);
> + return 0;
> +}
> +
> +static int config_input(AVFilterLink *link) {
Nit+: here and below:
static int config_input(AVFilterLink *link)
{
...
is favored (but I can fix it when committing, so no need to fix it
yourself if you don't care).
> + AlphaExtractContext *extract = link->dst->priv;
> + extract->is_packed_rgb =
> + ff_fill_rgba_map(extract->rgba_map, link->format) >= 0;
> + return 0;
> +}
> +
> +static void draw_slice(AVFilterLink *link, int y0, int h, int slice_dir)
> +{
> + AlphaExtractContext *extract = link->dst->priv;
> + AVFilterBufferRef *cur_buf = link->cur_buf;
> + AVFilterBufferRef *out_buf = link->dst->outputs[0]->out_buf;
> +
> + if (extract->is_packed_rgb) {
> + int x, y, pin, pout;
> + for (y = y0; y < (y0 + h); y++) {
> + for (x = 0; x < out_buf->video->w; x++) {
> + pin = y * cur_buf->linesize[0] + x * 4 + extract->rgba_map[A];
> + pout = y * out_buf->linesize[0] + x;
> + out_buf->data[0][pout] = cur_buf->data[0][pin];
> + }
> + }
> + } else {
> + const int linesize = cur_buf->linesize[A];
> + av_assert0(linesize == out_buf->linesize[Y]);
> +
> + memmove(out_buf->data[Y] + y0 * linesize,
> + cur_buf->data[A] + y0 * linesize,
> + linesize * h);
> + }
> + ff_draw_slice(link->dst->outputs[0], y0, h, slice_dir);
> +}
> +
> +AVFilter avfilter_vf_alphaextract = {
> + .name = "alphaextract",
> + .description = NULL_IF_CONFIG_SMALL("Extract an alpha channel as a "
> + "grayscale image component."),
> + .priv_size = sizeof(AlphaExtractContext),
> + .query_formats = query_formats,
> +
> + .inputs = (const AVFilterPad[]) {
> + { .name = "default",
> + .type = AVMEDIA_TYPE_VIDEO,
> + .config_props = config_input,
> + .draw_slice = draw_slice,
> + .min_perms = AV_PERM_READ },
> + { .name = NULL }
> + },
> + .outputs = (const AVFilterPad[]) {
> + { .name = "default",
> + .type = AVMEDIA_TYPE_VIDEO, },
> + { .name = NULL }
> + },
> +};
> diff --git a/libavfilter/vf_alphamerge.c b/libavfilter/vf_alphamerge.c
> new file mode 100644
> index 0000000..05f9f27
> --- /dev/null
> +++ b/libavfilter/vf_alphamerge.c
> @@ -0,0 +1,186 @@
> +/*
> + * Copyright (c) 2012 Steven Robertson
> + *
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +
> +/**
> + * @file
> + * copy an alpha component from another video's luma
> + */
> +
> +#include <string.h>
> +
> +#include "libavutil/pixfmt.h"
> +#include "avfilter.h"
> +#include "bufferqueue.h"
> +#include "drawutils.h"
> +#include "formats.h"
> +#include "internal.h"
> +#include "video.h"
> +
> +enum { Y, U, V, A };
> +
> +typedef struct {
> + int is_packed_rgb;
> + uint8_t rgba_map[4];
> + struct FFBufQueue queue_main;
> + struct FFBufQueue queue_alpha;
> +} AlphaMergeContext;
> +
> +static av_cold void uninit(AVFilterContext *ctx)
> +{
> + AlphaMergeContext *merge = ctx->priv;
> + ff_bufqueue_discard_all(&merge->queue_main);
> + ff_bufqueue_discard_all(&merge->queue_alpha);
> +}
> +
> +static int query_formats(AVFilterContext *ctx)
> +{
> + enum PixelFormat main_fmts[] = {
> + PIX_FMT_YUVA444P, PIX_FMT_YUVA422P, PIX_FMT_YUVA420P,
> + PIX_FMT_RGBA, PIX_FMT_BGRA, PIX_FMT_ARGB, PIX_FMT_ABGR,
> + PIX_FMT_NONE
> + };
> + enum PixelFormat alpha_fmts[] = { PIX_FMT_GRAY8, PIX_FMT_NONE };
> + AVFilterFormats *main_formats = ff_make_format_list(main_fmts);
> + AVFilterFormats *alpha_formats = ff_make_format_list(alpha_fmts);
> + ff_formats_ref(main_formats, &ctx->inputs[0]->out_formats);
> + ff_formats_ref(alpha_formats, &ctx->inputs[1]->out_formats);
> + ff_formats_ref(main_formats, &ctx->outputs[0]->in_formats);
> + return 0;
> +}
> +
> +static int config_input_main(AVFilterLink *link) {
> + AlphaMergeContext *merge = link->dst->priv;
> + merge->is_packed_rgb =
> + ff_fill_rgba_map(merge->rgba_map, link->format) >= 0;
> + return 0;
> +}
> +
> +static int config_output(AVFilterLink *outlink) {
> + AVFilterLink *inlink = outlink->src->inputs[0];
> + outlink->w = inlink->w;
> + outlink->h = inlink->h;
> + outlink->time_base = inlink->time_base;
> + outlink->sample_aspect_ratio = inlink->sample_aspect_ratio;
> + outlink->frame_rate = inlink->frame_rate;
> + return 0;
> +}
Is this required?
> +
> +static void start_frame(AVFilterLink *link, AVFilterBufferRef *picref) {}
> +static void draw_slice(AVFilterLink *link, int y, int h, int slice_dir) {}
> +
> +static void draw_frame(AVFilterContext *ctx,
> + AVFilterBufferRef *main_buf,
> + AVFilterBufferRef *alpha_buf)
> +{
> + AlphaMergeContext *merge = ctx->priv;
> + int h = main_buf->video->h;
> +
> + if (merge->is_packed_rgb) {
> + int x, y, pin, pout;
> + for (y = 0; y < h && y < alpha_buf->video->h; y++) {
> + for (x = 0; x < main_buf->video->w && x < alpha_buf->video->w; x++) {
> + pin = y * alpha_buf->linesize[0] + x;
> + pout = y * main_buf->linesize[0] + x * 4 + merge->rgba_map[A];
> + main_buf->data[0][pout] = alpha_buf->data[0][pin];
Nit: pin and pout are usually pointers rather than offsets, so you
could do:
uint8_t *pin, *pout;
...
pin = alpha_buf->data[0] + y * alpha_buf->linesize[0] + x;
pout = main_buf->data[0] + y * main_buf->linesize[0] + x * 4 + merge->rgba_map[A];
*pout = *pin;
or
main_buf->data[0][y * main_buf->linesize[0] + x * 4 + merge->rgba_map[A]] =
alpha_buf->data[0][y * alpha_buf->linesize[0] + x] =
or:
offin, offout;
...
> + }
> + }
> + } else {
> + int y;
> + int main_line = main_buf->linesize[A];
> + int alpha_line = alpha_buf->linesize[Y];
> +
> + for (y = 0; y < h && y < alpha_buf->video->h; y++) {
> + memmove(main_buf->data[A] + y * main_line,
> + alpha_buf->data[Y] + y * alpha_line,
> + FFMIN(main_line, alpha_line));
> + }
> + }
> + ff_draw_slice(ctx->outputs[0], 0, h, 1);
> +}
> +
> +static void end_frame(AVFilterLink *link) {
nit: link->inlink, helps readability/consistency
[...]
> diff --git a/tests/lavfi-regression.sh b/tests/lavfi-regression.sh
> index dd5e2da..224fc8f 100755
> --- a/tests/lavfi-regression.sh
> +++ b/tests/lavfi-regression.sh
> @@ -13,21 +13,25 @@ eval do_$test=y
>
> do_video_filter() {
> label=$1
> - filters=$2
> + filters="$2"
> shift 2
> printf '%-20s' $label
> run_avconv $DEC_OPTS -f image2 -vcodec pgmyuv -i $raw_src \
> $ENC_OPTS -vf "$filters" -vcodec rawvideo $* -f nut md5:
> }
>
> -do_lavfi() {
> - vfilters="slicify=random,$2"
> +do_lavfi_plain() {
> + vfilters="$2"
>
> if [ $test = $1 ] ; then
> do_video_filter $test "$vfilters"
> fi
> }
>
> +do_lavfi() {
> + do_lavfi_plain $1 "slicify=random,$2"
> +}
> +
> do_lavfi_colormatrix() {
> do_lavfi "${1}1" "$1=$4:$5,$1=$5:$3,$1=$3:$4,$1=$4:$3,$1=$3:$5,$1=$5:$2"
> do_lavfi "${1}2" "$1=$2:$3,$1=$3:$2,$1=$2:$4,$1=$4:$2,$1=$2:$5,$1=$5:$4"
> @@ -60,6 +64,11 @@ do_lavfi "vflip" "vflip"
> do_lavfi "vflip_crop" "vflip,crop=iw-100:ih-100:100:100"
> do_lavfi "vflip_vflip" "vflip,vflip"
>
> +do_lavfi_plain "alphamerge_rgb" "[in]slicify=random,format=bgra,split[o1][o2];[o1][o2]alphamerge[out]"
> +do_lavfi_plain "alphamerge_yuv" "[in]slicify=random,format=yuv420p,split[o1][o2];[o1][o2]alphamerge[out]"
> +do_lavfi_plain "alphaextract_rgb" "[in]slicify=random,format=bgra,split[o1][o2];[o1][o2]alphamerge,slicify=random,split[o3][o4];[o4]alphaextract[alpha];[o3][alpha]alphamerge[out]"
> +do_lavfi_plain "alphaextract_yuv" "[in]slicify=random,format=yuv420p,split[o1][o2];[o1][o2]alphamerge,slicify=random,split[o3][o4];[o4]alphaextract[alpha];[o3][alpha]alphamerge[out]"
The first part seems a duplicate of the first test, why not a simple:
do_lavfi_plain "alphaextract_rgb" "[in]slicify=random,split[o3][o4];[o4]alphaextract[alpha];[o3][alpha]alphamerge[out]"
?
[...]
Overall nice work.
--
FFmpeg = Fabulous and Fundamentalist Merciless Patchable Esoteric Guru
More information about the ffmpeg-devel
mailing list