[FFmpeg-devel] [PATCH v3 1/2] dxva: wait until D3D11 buffer copies are done before submitting them

[Steve Lhomme]

On 2020-08-11 8:58, Soft Works wrote:
> 
> 
>> -----Original Message-----
>> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
>> Steve Lhomme
>> Sent: Tuesday, August 11, 2020 6:35 AM
>> To: ffmpeg-devel at ffmpeg.org
>> Subject: Re: [FFmpeg-devel] [PATCH v3 1/2] dxva: wait until D3D11 buffer
>> copies are done before submitting them
> 
> [...]
> 
>>> But SetMultithreadProtected in fact has the effect of protecting the
>>> methods from ID3D11DeviceContext (not device).
>>> Like I explained several times already.
>>>
>>>
>>> Luckily I've just seen that there has been a change meanwhile.
>>> The undocumented behavior has finally been made public by adding a
>>> DX11 version of SetMultithreadProtected (which does the same as what
>>> happens when you use the DX10 version with DX11)
>>>
>>> https://docs.microsoft.com/en-us/windows/win32/api/d3d11_4/nn-
>> d3d11_4-
>>> id3d11multithread
>>
>> OK, this is very interesting. It certainly clarifies that it's ID3D11DeviceContext
>> related. The documentation is still sketchy though.
>> For example ::Enter() [1] says that it enters a device critical section, not a
>> device context critical section. So if you have multiple device context
>> everything is blocked. This kinda defeats the use of multiple threads.
>>
>> Also this interface is not available until Windows 10 Creator Update. So for
>> example it cannot be used on Windows 7. Maybe the D3D10 version has the
>> same effect but there's no guarantee.
> 
> Still playing "the doubtful" is not a sincere reaction after having been
> proven wrong.

Sorry if you seem to know all the answers already, but I don't and so I 
have to investigate.

>> I'll try without the mutex (old API) on Windows 7 and see the effect.

I did some more test on Win10 before trying Win7. And removing the mutex 
is not an option. There's a lot of bogus frames. It makes sense, as the 
API is not meant to be thread-safe.

The SubmitDecoderBuffers() has no way of knowing exactly which buffers 
you are providing. Each buffer is tied to an index of the slice so it 
could be called without a lock, as long as 2 threads don't try to use 
the same slice at the same time. But when you submit the buffers you 
only give the type. I suppose D3D11 takes the last buffer of each type 
that was released (which is pretty much an unmap, so should have similar 
internal synchronization). But if 2 threads release a buffer of the same 
type and submit it right after, D3D11 has no way to know which to use.

Here's an example:
- thread A releases a D3D11_VIDEO_DECODER_BUFFER_BITSTREAM for slice 0
- thread B releases a D3D11_VIDEO_DECODER_BUFFER_BITSTREAM for slice 1
- thread A submit a buffer of type D3D11_VIDEO_DECODER_BUFFER_BITSTREAM
- thread B submit a buffer of type D3D11_VIDEO_DECODER_BUFFER_BITSTREAM

D3D11 doesn't seem to keep track of which thread submitted what (and in 
single thread you could also release 2 buffer of the same type and 
submit them, I'm not sure what would happen). And so the Thread A submit 
is likely using the buffer from Thread B. I'm not even sure what Thread 
B would be using.

The mutex/lock groups the buffer get/release and submit together, so 
that no other thread can do the same thing at the same time. D3D11 
cannot do it by itself, even with multithread turned on. (D3D12 doesn't 
have this problem as you submit the actual buffer, not just the type [1][2])

None of that happens if you use a single thread for decoding.

> My patch requires feature level 11_1:
 >
> https://github.com/softworkz/ffmpeg_dx11/commit/c09cc37ce7f513717493e060df740aa0e7374257#diff-08dfe4cade908944fd2815bf36309d7bR591-R592
> 
> Windows 7 does not have DX11.1
> (there's a 11.1 platform update for Win 7 but it doesn't provide full 11.1)
> For Win7 and below we're using D3D9 with DXVA2 only.
> 
> Kind regards,
> softworkz
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel

[1] 
https://docs.microsoft.com/en-us/windows/win32/api/d3d12video/nf-d3d12video-id3d12videodecodecommandlist-decodeframe
[2] 
https://docs.microsoft.com/en-us/windows/win32/api/d3d12video/ns-d3d12video-d3d12_video_decode_frame_argument