[FFmpeg-devel] [PATCH] slicethread: Limit the automatic number of threads to 16

[Andreas Rheinhardt]

Lukas Fellechner:
>> Gesendet: Montag, 05. September 2022 um 21:58 Uhr
>> Von: "Martin Storsjö" <martin at martin.st>
>> An: ffmpeg-devel at ffmpeg.org
>> Betreff: Re: [FFmpeg-devel] [PATCH] slicethread: Limit the automatic number of threads to 16
>> On Mon, 5 Sep 2022, Martin Storsjö wrote:
>>
>>> This matches a similar cap on the number of automatic threads
>>> in libavcodec/pthread_slice.c.
>>>
>>> On systems with lots of cores, this does speed things up in
>>> general (measurable on the level of the runtime of running
>>> "make fate"), and fixes a couple fate failures in 32 bit mode on
>>> such machines (where spawning a huge number of threads runs
>>> out of address space).
>>> ---
>>
>> On second thought - this observation that it speeds up "make -j$(nproc)
>> fate" isn't surprising at all; as long as there are jobs to saturate all
>> cores with the make level parallelism anyway, any threading within each
>> job just adds extra overhead, nothing more.
>>
>> // Martin
> 
> Agreed, this observation of massively parallel test runs does not tell
> much about real world performance.
> There are really two separate issues here:
> 
> 1. Running out of address space in 32-bit processes
> 
> It probably makes sense to limit auto threads to 16, but it should only
> be done in 32-bit processes. A 64-bit process should never run out of
> address space. We should not cripple high end machines running
> 64-bit applications.
> 
> 
> Sidenotes about "it does not make sense to have more than 16 slices":
> 
> On 8K video, when using 32 threads, each thread will process 256 lines
> or about 1MP (> FullHD!). Sure makes sense to me. But even for sw decoding
> 4K video, having more than 16 threads on a powerful machine makes sense.
> 
> Intel's next desktop CPUs will have up to 24 physical cores. The
> proposed change would limit them to use only 16 cores, even on 64-bit.
> 

This part is completely wrong: You can always set the number of threads
manually.
(Btw: 1. 8K is the horizontal resolution; the vertical resolution of it
is 4360 (when using 16:9), so every thread processes 135 lines which
have as many pixels as 540 lines of FullHD. 2. FullHD has about 2MP.)

- Andreas