[FFmpeg-devel] [PATCH][RFC] avcodec: disallow hwaccel with frame threads

[Gwenole Beauchesne]

2015-10-21 11:27 GMT+02:00 Hendrik Leppkes <h.leppkes at gmail.com>:
> On Sat, Oct 17, 2015 at 9:28 PM, Hendrik Leppkes <h.leppkes at gmail.com> wrote:
>> On Sat, Oct 17, 2015 at 9:27 PM, Hendrik Leppkes <h.leppkes at gmail.com> wrote:
>>> ---
>>>  libavcodec/utils.c | 6 ++++++
>>>  1 file changed, 6 insertions(+)
>>>
>>
>> Above patch is submitted as an RFC, however I strongly believe its the
>> only way to keep hwaccels sane in the future.
>> There are several known problems when a hwaccel is used with frame
>> threading, at least with DXVA2, ranging from simple image corruption
>> to crashes in the GPU drivers.
>>
>> All the problems essentially come down to two factors:
>>
>> (1) while avcodec tries to prevent simultaneous access from different
>> threads, it cannot control user code.
>> As a API user you have no chance to avoid simultaneous access to the
>> hardware surfaces with frame threading, because as soon as avcodec
>> hands you a decoded surface,
>> it'll already have started working on the next one. This is a common
>> source for image corruption, as the decoder may fail to get a
>> reference frame if its currently
>> locked by the user code.
>>
>> This issue is not really fixable, other than introducing a mutex
>> around every call that the user code would have to lock as well.
>> API break and making hwaccel even more complex to use.
>>
>> (2) decoders often have had (or still have) trouble initializing the
>> hwaccel properly with multiple threads, which can result in multiple
>> threads re-creating the hwaccel
>> and disrupting decoding, or even crashing in the driver.
>>
>> While this is in theory fixable inside the decoders in question, it
>> can lead to quite a bit of complexity, which should be avoided.
>>
>> We should not be offering a mode that is known to be broken and even
>> crash (even if the crash is not in our code).
>> On top of that, frame threading with hwaccel does not actually improve
>> the speed at all, if anything it might be slightly slower, and will
>> definitely use more GPU memory.
>>
>> The only reason this combination even exists is because API users
>> insisted on it, and it was then added without properly taking the
>> consequences into account.
>>
>> There is an easy solution to keep using frame threads with software
>> decoding while using a hwaccel otherwise - its used by myself, by mpv
>> and by Kodi - simply re-open the decoder with the new threading flags.
>> Its really not a lot of work, and keeps the code sane and the decoding reliable.
>>
>
> No further comments? No thoughts? No-one that would object if I were
> to push the proposed patch?

Fine with me.

Probably add a link to your ffmpeg-devel message since it provides all
the necessary rationale behind this?

Thanks,
-- 
Gwenole Beauchesne
Intel Corporation SAS / 2 rue de Paris, 92196 Meudon Cedex, France
Registration Number (RCS): Nanterre B 302 456 199