[FFmpeg-devel] [PATCH] libavfilter/scale2ref: Add constants for the primary input

[James Almer]

On 6/1/2017 6:37 PM, Michael Niedermayer wrote:
> On Tue, May 30, 2017 at 01:34:29PM -0400, Kevin Mark wrote:
>> Variables pertaining to the main video are now available when
>> using the scale2ref filter. This allows, as an example, scaling a
>> video with another as a reference point while maintaining the
>> original aspect ratio of the primary/non-reference video.
>>
>> Consider the following graph: scale2ref=iw/6:-1 [main][ref]
>> This will scale [main] to 1/6 the width of [ref] while maintaining
>> the aspect ratio. This works well when the AR of [ref] is equal to
>> the AR of [main] only. What the above filter really does is
>> maintain the AR of [ref] when scaling [main]. So in all non-same-AR
>> situations [main] will appear stretched or compressed to conform to
>> the same AR of the reference video. Without doing this calculation
>> externally there is no way to scale in reference to another input
>> while maintaining AR in libavfilter.
>>
>> To make this possible, we introduce eight new constants to be used
>> in the w and h expressions only in the scale2ref filter:
>>
>>  * main_w/main_h: width/height of the main input video
>>  * main_a: aspect ratio of the main input video
>>  * main_sar: sample aspect ratio of the main input video
>>  * main_dar: display aspect ratio of the main input video
>>  * main_hsub/main_vsub: horiz/vert chroma subsample vals of main
>>  * mdar: a shorthand alias of main_dar
>>
>> Of course, not all of these constants are needed for maintaining the
>> AR, but adding additional constants in line of what is available for
>> in/out allows for other scaling possibilities I have not imagined.
>>
>> So to now scale a video to 1/6 the size of another video using the
>> width and maintaining its own aspect ratio you can do this:
>>
>> scale2ref=iw/6:ow/mdar [main][ref]
>>
>> This is ideal for picture-in-picture configurations where you could
>> have a square or 4:3 video overlaid on a corner of a larger 16:9
>> feed all while keeping the scaled video in the corner at its correct
>> aspect ratio and always the same size relative to the larger video.
>>
>> I've tried to re-use as much code as possible. I could not find a way
>> to avoid duplication of the var_names array. It must now be kept in
>> sync with the other (the normal one and the scale2ref one) for
>> everything to work which does not seem ideal. For every new variable
>> introduced/removed into/from the normal scale filter one must be
>> added/removed to/from the scale2ref version. Suggestions on how to
>> avoid var_names duplication are welcome.
>>
>> var_values has been increased to always be large enough for the
>> additional scale2ref variables. I do not forsee this being a problem
>> as the names variable will always be the correct size. From my
>> understanding of av_expr_parse_and_eval it will stop processing
>> variables when it runs out of names even though there may be
>> additional (potentially uninitialized) entries in the values array.
>> The ideal solution here would be using a variable-length array but
>> that is unsupported in C90.
>>
>> This patch does not remove any functionality and is strictly a
>> feature patch. There are no API changes. Behavior does not change for
>> any previously valid inputs.
>>
>> The applicable documentation has also been updated.
>>
>> Signed-off-by: Kevin Mark <kmark937 at gmail.com>
>> ---
>>  doc/filters.texi    | 26 ++++++++++++++++++-
>>  libavfilter/scale.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++---
>>  2 files changed, 93 insertions(+), 5 deletions(-)
> 
> applied
> 
> can you add a fate test that uses the new identifers ?
> 
> Thanks

This broke 820 tests with Valgrind. All of them seem to point to

==1253== Invalid read of size 8
==1253==    at 0x697ECA: ff_scale_eval_dimensions (scale.c:118)