[FFmpeg-devel] [RFC] Affine video transformation filter

[Michael Bradshaw]

On Thu, Jul 5, 2012 at 5:23 AM, Stefano Sabatini <stefasab at gmail.com> wrote:
> While an affine transform would cover many already existing and more
> specialized filters, it would be generally less efficient *and* with
> far less options (consider for example all the many options and
> formats supported by libswscale).
>
> So it is fine to have more specialized filters, provided that they're
> simpler, more efficient, easier to use than a more generalized filter.

I feel the same.

>> While I've got your attention, what seems like a sane way to pass
>> arguments to an affine transformation filter? I'm thinking something
>> along the lines of
>> "affine=scale=1.1\,1.3:rotate=15:translate=4\,10:shear=1.2\,2:reflect=-1\,0:sampling=bilinear"
>> for example where order matters and options can be repeated (and maybe
>> allowing "affine=matrix=x11:x12:x13:x21:x22:x23:x31:x32:x33"). I
>
> If you want my opinion we should support matrix parsing (matlab
> notation may do), then you can wrap more specialized filters
> (e.g. rotate=theta) by composing the matrix. Matrix parsing may be
> useful in ffmpeg.c and for other filters (e.g. a generic convolution
> filter).

I like the idea of a more generalized matrix parsing functionality. I
don't know if it would be good to allow for all of matlab's notations
for creating a matrix, but I think a standardized one would be useful.

Would it be going to far to allow the user to specify an equation for
a matrix entry (e.g. some entry could depend on input video
width/height)?

>> haven't thought of a way to specify the background fill color that I
>> like, though.
>
> Do you mean you don't know which *syntax* or how to implement it? For
> the syntax I suppose we should go with named options, for code usage
> we have the internal drawutils which should somehow help.

A bit of both, actually. I thought of named options, but I also like
the idea of allowing the user to specify any color choice (maybe as a
hex #rrggbbaa). I'll think about it some more. drawutils looks like it
should be helpful, though.

> Also check for ideas:
>
> - my rotate filter (supports fill color and dynamic rotation
>   controlled by an expression, integer-only processing -> fast, but
>   which had a somehow serious problem so was never committed)

That has already given me some ideas. I'd like to look over it more,
but all I can find about it is in this thread here:
https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2011-May/111413.html
and it's missing a significant earlier portion. Do you know where I
can find the full patch to look over? And what were the serious
problems with it (so I can avoid them)?

> - libavfilter/transform.h which already implements an API for affine
>   transforms (used in deshake)

I did look at that but it doesn't support some things like shearing,
reflection, different x/y scaling... I've thought of adding them, but
to me the only sensible way to add all of them is to start with an
identity matrix and then use separate functions for each separate
operation (and/or maybe  have get functions that return corresponding
matrices (like a get_scale_matrix(float sx, float sy, float* mat) and
just concatenate them), composing your own matrix instead of calling
one super function.

>> Your thoughts are appreciated.
>
> Note also that I'm personally fine with a more specialized filter
> (e.g. a rotate filter) if that's your objective, since I know that
> designing a generic filter which is both fast and flexible is much
> harder.

A rotation filter just sounded like a fun project, but I've become
more interested in this. I don't know if I'll ever actually do a
rotation filter.

Thanks,

Michael