[FFmpeg-devel] [PATCH] pthread: Fix ff_thread_get_formatissues when called outside frame decode.

[Hendrik Leppkes]

On Mon, Mar 9, 2015 at 3:28 PM, Hendrik Leppkes <h.leppkes at gmail.com> wrote:
> On Mon, Mar 9, 2015 at 1:50 PM, Reimar Döffinger
> <Reimar.Doeffinger at gmx.de> wrote:
>> On 9 March 2015 13:34:24 CET, Hendrik Leppkes <h.leppkes at gmail.com> wrote:
>>>On Mon, Mar 9, 2015 at 7:38 AM, Rainer Hochecker
>>><fernetmenta at online.de> wrote:
>>>> Reimar Döffinger <Reimar.Doeffinger <at> gmx.de> writes:
>>>>
>>>>>
>>>>> Any reason to believe this patch causes it?
>>>>> Because I can't see how it would.
>>>>> Maybe it's just a bug with DXVA and multithreading in the HEVC code?
>>>>> Can you provide some more information like a stacktrace, possibly
>>>using a
>>>> tool like DrMemory?
>>>>
>>>> I don't think the patch itself is the root cause of the issue, it
>>>just
>>>> triggers it. get_format is called 4 times, something seems to get
>>>corrupted
>>>> in opening our hw decoder.
>>>> Do you have an explanation why it works with thread_safe_callbacks
>>>set to 1?
>>>>
>>>> I am wondering if hevc misses the multithreading fix done for other
>>>codecs:
>>>> http://ffmpeg.org/pipermail/ffmpeg-cvslog/2013-March/062620.html
>>>>
>>>> What do you think?
>>>>
>>>
>>>I build the HEVC HWAccel support, and I consider frame-threading with
>>>hwaccel an abomination that should not be supported, so I didn't spend
>>>any time thinking about it.
>>>
>>>Honestly, there are several issues with MT+HWaccel in H264 as well, at
>>>least on Intel hardware, so I can only strongly urge everyone to not
>>>combine it. There is zero advantages, and only downsides!
>>
>> Only if you completely ignore the practical issues that anyone trying to use it will have.
>> In many cases applications will not know whether hardware supports decoding the video until there get_format call. Worse, it can even change from supported to not supported.
>> Do you expect everyone to disable multithreading "just in case"? Buffer the last n bits of data to be able to restart decoding?
>
> What I do is simply restart decoding with the packet that failed the
> hardware decoder. Don't need to buffer anything, you still have the
> AVPacket in question anyway.
> This allows a perfect and seamless change without requiring the hwaccel-mt hack.

In the end it also comes down to this:
On Intel hardware, MT+HWAccel (at least DXVA2) is fundamentally
broken, and unfixable in the current design. The problem is that
concurrent access to the decoded surfaces from different threads
simply does not work, and while avcodec locks the decoding threads out
that only ever one is working, it does not know when the user code
accesses the surface, and there is no way to tell it either. So
anytime a frame is used as a reference and you try to process it in
user code at the same time, decoding fails. This can be reproduced
with ffmpeg CLI even.

This reason is enough to consider the entire concept flawed, imho.

Of course avcodec shouldn't crash when you use it, but a patch to
disallow this combination would never be accepted by yourself and some
others that "like" this cheap (albeit wrong) method for fallback.
So the only solution is to add the extra magic that makes it not
crash, and someone should produce a patch for that if their setup can
reproduce it.

- Hendrik