[FFmpeg-devel] [PATCH] opt: check image size when setting it

[Andreas Cadhalpun]

On 11.12.2016 21:03, Hendrik Leppkes wrote:
> On Sun, Dec 11, 2016 at 7:48 PM, Andreas Cadhalpun
> <andreas.cadhalpun at googlemail.com> wrote:
>> On 11.12.2016 10:04, Hendrik Leppkes wrote:
>>> I still see the problem that this option code does not know which
>>> pix_fmt the numbers relate to and as such would keep the old limit in
>>> place despite there being no technical reasons for it.
>>
>> And I still think that av_image_check_size should be changed to
>> accept the largest value valid for any pixel format (once every place
>> needing stricter limits is switched to the pixel format sensitive
>> check).
>> Do you disagree with this or what is your point?
> 
> I could probably live with a simple w*h overflow check (more or less),
> which av_image_check_size then probably would end up being if I
> understand you right?

I don't think so. For example, av_image_check_size2 accepts resolutions
like 100000x80000 for AV_PIX_FMT_MONOWHITE and thus av_image_check_size
should also accept this, even though the number of pixels is larger
than INT_MAX. However, that's not the current state of affairs, so
until the work is done to actually use the pixel format specific limits,
the option code should check for the old limit.

> Thats much higher then the current limit in most cases and while we
> should try to move this to size_t/ptrdiff_t eventually to lift the
> limit even higher on 64-bit platforms, its OK for now.

Note that av_image_check_size is documented to check that
"all bytes of the image can be addressed with a signed int",
so increasing the limit higher requires using a different function.

Best regards,
Andreas