[FFmpeg-devel] [PATCH] avutil/file_open: avoid file handle inheritance on Windows

[Tobias Rapp]

On 30.10.2015 11:54, Hendrik Leppkes wrote:
> On Fri, Oct 30, 2015 at 11:30 AM, Tobias Rapp <t.rapp at noa-archive.com> wrote:
>> On 29.10.2015 09:38, Tobias Rapp wrote:
>>>
>>> Attached patch fixes file lock issues in my Windows application when a
>>> child process is started with handle inheritance enabled (standard
>>> input/output redirection) while a FFmpeg transcoding is running in the
>>> parent process.
>>>
>>> BTW: It would be great if the patch could also be applied to the 2.7/2.8
>>> release branches.
>>
>>
>> Forgot to add links relevant to the subject.
>>
>> [1] https://msdn.microsoft.com/en-us/library/w7sa2b22.aspx
>>
>> Describes the _wsopen() function and the O_NOINHERIT flag. File handles
>> opened by _wsopen() are inheritable by default.
>>
>> [2]
>> https://msdn.microsoft.com/en-us/library/windows/desktop/aa363858%28v=vs.85%29.aspx
>>
>> Describes the CreateFile() function. Not directly relevant here, often used
>> in cases outside of C/libc. Opens file without inheritance by default
>> (lpSecurityAttributes is NULL).
>>
>> [3]
>> https://msdn.microsoft.com/en-us/library/windows/desktop/ms682425%28v=vs.85%29.aspx
>>
>> Describes handle inheritance when creating new processes. Handle inheritance
>> must be enabled (bInheritHandles = TRUE) e.g. when you want to pass handles
>> for stdin/stdout via lpStartupInfo.
>>
>
> Its probably fine, the only concerns I can think of is what happens if
> you try use ffmpeg.exe as a subprocess operating on a pipe fed by the
> parent process, would this still work?

Yes, in that case you can still create your pipe handle and explicitly 
set bInheritHandle = TRUE in the security attributes. Then pass the 
handle to ffmpeg.exe. This is basically what I'm currently doing except 
that I pass the pipe handles as stdin/stdout to the client ffmpeg.exe 
process instead of using them within some "pipe:1234" protocol URL.

Regards,
Tobias