[FFmpeg-devel] [PATCH] avdevice/decklink: adjust for timecode lag

[Gyan]

On 31-08-2019 08:11 PM, Marton Balint wrote:
>
>
> On Sat, 31 Aug 2019, Gyan wrote:
>
>>
>>
>> On 31-08-2019 04:14 AM, Marton Balint wrote:
>>>
>>>
>>> On Fri, 30 Aug 2019, Gyan wrote:
>>>
>>>>
>>>>
>>>> On 22-08-2019 01:10 AM, Marton Balint wrote:
>>>>>
>>>>>
>>>>> On Tue, 20 Aug 2019, Devin Heitmueller wrote:
>>>>>
>>>>>>> A couple of follow-up Qs:
>>>>>>>
>>>>>>> Is auto-detection available for all Decklink devices?
>>>>>>
>>>>>> No, but AFAIK it is for all devices which support SDI. Generally 
>>>>>> it's
>>>>>> the older analog capture devices which don't support it.
>>>>>>
>>>>>>> For those for which it is available, are there any edge cases in 
>>>>>>> which
>>>>>>> it sets inaccurate mode?
>>>>>>
>>>>>> I don't trust the existing detection code enough to use it in
>>>>>> production.  It often fails to detect and thus ffmpeg will exit at
>>>>>> startup.
>>>>>
>>>>> Can this be fixed by simply increasing the timeout from 1 sec to 2 
>>>>> seconds?
>>>>>
>>>>>> Also, there are cases where it will misdetect 1080i59 as
>>>>>> 1080p30 depending on the card.  It's been on my TODO list for a 
>>>>>> while
>>>>>> to make that code more robust (I believe I know what most of the
>>>>>> issues are), but it hasn't been critical for any of my use cases.
>>>>>
>>>>> Hmm, interesting... When you say the issue is card-dependant, does 
>>>>> this mean card _model_ dependant or that the issue can affect one 
>>>>> card and not the other with of the same model/fw?
>>>>
>>>> If an user can't or won't use format auto detection, then there is 
>>>> no way to reliably obtain the correct timecode without this patch 
>>>> or some similar hack.
>>>>
>>>> Maybe the offset-related ops could be guarded by a check for auto 
>>>> detection, even though the patch won't hurt even if it was used in 
>>>> that case.
>>>
>>> I am still not liking this too much because we are guessing some 
>>> timecode which we never actually saw, and the guessing is based on 
>>> the number of the VideoFrameArrived callbacks. Yes, usually it is 
>>> called once for every frame, but some frames might not contain 
>>> audio, also there are some examples in the SDK docs where only an 
>>> audio frame (and no video frame) is available in the callback.
>>
>> Yes, that's why there is a new counter, incremented only upon getting 
>> a video frame and reset until the demuxer successfully queues the 
>> packet of the first video frame. I just noticed an issue with it, but 
>> I can change that.
>>
>>
>>> So it is up to the receiving application to do something to keep 
>>> audio and video in sync, it either drops/duplicates video or audio. 
>>> If it drops/dups video then our guessed start timecode can be wrong 
>>> even if we were only counting the video frames.
>>
>> By receiving app, do you mean the Decklink driver   or 
>> decklink_dec/ffmpeg.c?
>
> ffmpeg.c
>
>>
>>
>>> So I think that if a workaround/hack is needed, we should do that 
>>> differently. One idea is to introduce an option which makes the code 
>>> drop video till the first valid video frame, or the first valid 
>>> timecode. But if autodetection works (or it can be fixed), then I 
>>> just don't see too much use even for this.
>>
>> As Ilinca's tests showed, without auto format detection, the stored 
>> timecode is *wrong*. That's a bug so some change is *required*.
>
> The bug is that decklink_dec sets a timecode to AVStream metadata 
> which is not the timecode of the first frame. As far as I recall when 
> accepting the decklink timecode support patch, I pointed out that it 
> is not very useful because it might not be the TC of the first frame.
>
> I am fine with a patch which only sets AVStream timecode if it is 
> indeed the first frame. I am fine with a patch which drops starter 
> frames without timecode when some option is set. I am not fine with a 
> patch which does calculations in deckink_dec to guess a starter timecode.

Ok. Will get back.

Thanks,
Gyan