[FFmpeg-devel] [PATCH v2 1/3] libavutil/imgutils: add utility to get plane sizes

[James Almer]

On 7/10/2020 9:03 AM, Nicolas George wrote:
> James Almer (12020-07-09):
>> int av_image_fill_plane_sizes(size_t *size, size_t planesizes[4], enum
>> AVPixelFormat pix_fmt, int height, const ptrdiff_t linesizes[4])
>>
>> But it's also somewhat ugly and inconsistent with other functions in
>> this module.
> 
> I find this is by far the best solution. Even more so since the total
> size is not the main output of the function as specified by its name. It
> is also more future-proof, and allows potential consistency for cases
> that return a pointer.
> 
> I would be curious to know what you find ugly with this; I am frequently
> baffled by similar judgements of yours. Would you try to analyze it?
> 
> Regards,

It's adding an extra parameter to get a value that ultimately can be
derived from the output of another parameter. If you can use the >0 part
of the return value for that, then to me it sounds ideal.
It's not like the functions that allocate a struct, where if you return
the pointer instead of passing it as an output parameter you lose the
ability to signal errors. In those this method is nicer.

I'd very much prefer to keep this new function consistent with either
av_image_fill_linesizes() (Output array as parameter, no sum of sizes as
return value), or av_image_fill_pointers() (Output array as parameter,
sum of sizes as return value).

Also, we could schedule changing all linesize parameters to ptrfdiff_t
on the next bump.