[FFmpeg-devel] [PATCH v2 1/3] decklink: Don't take for granted that first frame to decklink output will be PTS 0

[Marton Balint]

On Fri, 3 Mar 2023, Devin Heitmueller wrote:

> The existing code assumed that the first frame received by the decklink
> output would always be PTS zero.  However if running in other timing
> modes than the default of CBR, items such as frame dropping at the
> beginning may result in starting at a non-zero PTS.
>
> For example, in our setup because we discard probing data and run
> with "-vsync 2" the first video frame scheduled to the decklink
> output will have a PTS around 170.  Scheduling frames too far into
> the future will either fail or cause a backlog of frames scheduled
> far enough into the future that the entire pipeline will stall.
>
> Issue can be reproduced with the following command-line:
>
> ./ffmpeg -copyts -i foo.ts -f decklink -vcodec v210 -ac 2  'DeckLink Duo (4)'
>
> Keep track of the PTS of the first frame received, so that when
> we enable start playback we can provide that value to the decklink
> driver.
>
> Thanks to Marton Balint for review and suggestion to use
> AV_NOPTS_VALUE rather than zero for the initial value.
>
> Signed-off-by: Devin Heitmueller <dheitmueller at ltnglobal.com>
> ---
> libavdevice/decklink_common.h | 1 +
> libavdevice/decklink_enc.cpp  | 8 ++++++--
> 2 files changed, 7 insertions(+), 2 deletions(-)

Thanks, will apply the series.

Regards,
Marton

>
> diff --git a/libavdevice/decklink_common.h b/libavdevice/decklink_common.h
> index 79d6ac5b38..088e165ee7 100644
> --- a/libavdevice/decklink_common.h
> +++ b/libavdevice/decklink_common.h
> @@ -118,6 +118,7 @@ struct decklink_ctx {
>
>     /* Status */
>     int playback_started;
> +    int64_t first_pts;
>     int64_t last_pts;
>     unsigned long frameCount;
>     unsigned int dropped;
> diff --git a/libavdevice/decklink_enc.cpp b/libavdevice/decklink_enc.cpp
> index fb686b9032..5a435ddc78 100644
> --- a/libavdevice/decklink_enc.cpp
> +++ b/libavdevice/decklink_enc.cpp
> @@ -486,6 +486,9 @@ static int decklink_write_video_packet(AVFormatContext *avctx, AVPacket *pkt)
>     ctx->frames_buffer_available_spots--;
>     pthread_mutex_unlock(&ctx->mutex);
>
> +    if (ctx->first_pts == AV_NOPTS_VALUE)
> +        ctx->first_pts = pkt->pts;
> +
>     /* Schedule frame for playback. */
>     hr = ctx->dlo->ScheduleVideoFrame((class IDeckLinkVideoFrame *) frame,
>                                       pkt->pts * ctx->bmd_tb_num,
> @@ -505,14 +508,14 @@ static int decklink_write_video_packet(AVFormatContext *avctx, AVPacket *pkt)
>                " Video may misbehave!\n");
>
>     /* Preroll video frames. */
> -    if (!ctx->playback_started && pkt->pts > ctx->frames_preroll) {
> +    if (!ctx->playback_started && pkt->pts > (ctx->first_pts + ctx->frames_preroll)) {
>         av_log(avctx, AV_LOG_DEBUG, "Ending audio preroll.\n");
>         if (ctx->audio && ctx->dlo->EndAudioPreroll() != S_OK) {
>             av_log(avctx, AV_LOG_ERROR, "Could not end audio preroll!\n");
>             return AVERROR(EIO);
>         }
>         av_log(avctx, AV_LOG_DEBUG, "Starting scheduled playback.\n");
> -        if (ctx->dlo->StartScheduledPlayback(0, ctx->bmd_tb_den, 1.0) != S_OK) {
> +        if (ctx->dlo->StartScheduledPlayback(ctx->first_pts * ctx->bmd_tb_num, ctx->bmd_tb_den, 1.0) != S_OK) {
>             av_log(avctx, AV_LOG_ERROR, "Could not start scheduled playback!\n");
>             return AVERROR(EIO);
>         }
> @@ -559,6 +562,7 @@ av_cold int ff_decklink_write_header(AVFormatContext *avctx)
>     ctx->list_formats = cctx->list_formats;
>     ctx->preroll      = cctx->preroll;
>     ctx->duplex_mode  = cctx->duplex_mode;
> +    ctx->first_pts    = AV_NOPTS_VALUE;
>     if (cctx->link > 0 && (unsigned int)cctx->link < FF_ARRAY_ELEMS(decklink_link_conf_map))
>         ctx->link = decklink_link_conf_map[cctx->link];
>     cctx->ctx = ctx;
> -- 
> 2.35.1.655.ga68dfadae5
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>