[FFmpeg-devel] Patch for libx265 memory leak

[Timo Rothenpieler]

On 16.11.2024 19:46, Tom Vaughan wrote:
> Any further details you or others could provide to help the x265 dev team resolve the issue would be appreciated.
> 
> Can you share your steps to reproduce this issue? What parameters were changed? Max CTU size?
> 
> Is this documentation inadequate? https://x265.readthedocs.io/en/master/api.html#build-considerations

Well, it documents that it'll initialize global variables, but that 
doesn't make it better.
FFmpeg doesn't check that. Nor sure if it even can?
So depending on what the user does, it could just crash if the CTU size 
mismatches between parallel encodes.

So the libx265.c wrapper at least would need to check the already 
initialized global CTU state, and cleanly error out if it mismatches.
Or plain not allow more than exactly one x265 encode.

> On 11/11/24, 10:46 PM, "ffmpeg-devel on behalf of Damiano Galassi" <ffmpeg-devel-bounces at ffmpeg.org <mailto:ffmpeg-devel-bounces at ffmpeg.org> on behalf of damiog at gmail.com <mailto:damiog at gmail.com>> wrote:
> 
> 
> On Tue, Nov 12, 2024 at 3:38 AM Zhao Zhili <quinkblack at foxmail.com <mailto:quinkblack at foxmail.com>> wrote:
> 
> 
>>
>> cleanup() release library static allocations, and I don’t see lock or
>> reference count
>> within x265_cleanup(). So call cleanup() will break other x265 encoder
>> instance,
>> right?
> 
> 
> 
> 
> x265 already crashes when trying to run two encodes with different settings
> in
> the same process, because it's storing some instance specific data in
> global variables
> that are overwritten when starting a new encode.
> 
> 
> So at least with this patch it won't leaks the global data,
> but the root issue needs to be fixed in x265.
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org <mailto:ffmpeg-devel at ffmpeg.org>
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel <https://ffmpeg.org/mailman/listinfo/ffmpeg-devel>
> 
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org <mailto:ffmpeg-devel-request at ffmpeg.org> with subject "unsubscribe".
> 
> 
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".