[FFmpeg-devel] [PATCH v12 1/8] libavcodec/decode.c: intercept `AV_PKT_DATA_METADATA_UPDATE` packet extra data, attach them to the next decoded frame.

[Michael Niedermayer]

On Tue, Apr 15, 2025 at 05:22:29PM -0500, Romain Beauxis wrote:
> ---
>  libavcodec/decode.c | 19 +++++++++++++++++++
>  1 file changed, 19 insertions(+)
> 
> diff --git a/libavcodec/decode.c b/libavcodec/decode.c
> index fca0c7ff58..06d899a9dd 100644
> --- a/libavcodec/decode.c
> +++ b/libavcodec/decode.c
> @@ -97,6 +97,8 @@ typedef struct DecodeContext {
>      int lcevc_frame;
>      int width;
>      int height;
> +
> +    AVDictionary *pending_metadata;
>  } DecodeContext;
>  
>  static DecodeContext *decode_ctx(AVCodecInternal *avci)
> @@ -702,6 +704,8 @@ int attribute_align_arg avcodec_send_packet(AVCodecContext *avctx, const AVPacke
>  {
>      AVCodecInternal *avci = avctx->internal;
>      DecodeContext     *dc = decode_ctx(avci);
> +    const uint8_t *side_metadata;
> +    size_t size;
>      int ret;
>  
>      if (!avcodec_is_open(avctx) || !av_codec_is_decoder(avctx->codec))
> @@ -719,6 +723,14 @@ int attribute_align_arg avcodec_send_packet(AVCodecContext *avctx, const AVPacke
>          ret = av_packet_ref(avci->buffer_pkt, avpkt);
>          if (ret < 0)
>              return ret;
> +
> +        side_metadata = av_packet_get_side_data(avpkt, AV_PKT_DATA_METADATA_UPDATE, &size);
> +        if (side_metadata) {
> +            av_dict_free(&dc->pending_metadata);
> +            ret = av_packet_unpack_dictionary(side_metadata, size, &dc->pending_metadata);
> +            if (ret < 0)
> +                return ret;
> +        }
>      } else
>          dc->draining_started = 1;
>  
> @@ -788,6 +800,7 @@ fail:
>  int ff_decode_receive_frame(AVCodecContext *avctx, AVFrame *frame)
>  {
>      AVCodecInternal *avci = avctx->internal;
> +    DecodeContext     *dc = decode_ctx(avci);
>      int ret;
>  
>      if (avci->buffer_frame->buf[0]) {
> @@ -810,6 +823,11 @@ int ff_decode_receive_frame(AVCodecContext *avctx, AVFrame *frame)
>  
>      avctx->frame_num++;
>  
> +    if (dc->pending_metadata) {
> +        av_dict_copy(&frame->metadata, dc->pending_metadata, AV_DICT_APPEND);
> +        av_dict_free(&dc->pending_metadata);
> +    }
> +
>      return 0;
>  fail:
>      av_frame_unref(frame);
> @@ -2220,4 +2238,5 @@ void ff_decode_internal_uninit(AVCodecContext *avctx)
>      DecodeContext *dc = decode_ctx(avci);
>  
>      av_refstruct_unref(&dc->lcevc);
> +    av_dict_free(&dc->pending_metadata);
>  }
> -- 
> 2.39.5 (Apple Git-154)

My issue with this is the same as it was previously
Please correct me if iam wrong (it is very possible that iam wrong on
some assumtations here)

the decoder is variable delay (exact delay depends on cpu cores and luck)
on the input we have some side data from an AVPacket
then some packets will result in nothing to come out (because maybe they are damaged
or the users asked non key frames to be skiped or whatever)
some packets will decode into AVFrames in one of several threads
and these AVFrames then come back out

how can code that seems to ignore this be correct ?

Shouldnt the data be associated with the packet  go through the
decoder with it ?
Or is this data not associated with the packet in that sense ?

thx


[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The educated differ from the uneducated as much as the living from the
dead. -- Aristotle 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20250420/6d978148/attachment.sig>