[FFmpeg-devel] [PATCH v9 3/5] avcodec/libjxl: add Jpeg XL encoding via libjxl

[Andreas Rheinhardt]

Leo Izen:
> 
> On 3/30/22 07:12, Andreas Rheinhardt wrote:
>> buffer_size is not used after libjxl_encode_close() at all, as this is
>> the codec's close callback; libjxl_encode_frame() meanwhile may be
>> called again after freeing and if it reaches JxlEncoderProcessOutput(),
>> the latter would be called with next_out == NULL and available != 0.
>> JxlEncoderProcessOutput() might check for this (and error out) or it may
>> just crash.
>>
> Can encode2 really be called again if it returns with AVERROR(ENOMEM)? I
> was under the impression that this preventing further calls.
> 

Why should this be so? (E.g. a user might close other processes in order
to free up memory upon receiving AVERROR(ENOMEM) and try again.)

- Andreas