[FFmpeg-devel] [PATCH 03/11] avformat/avformat: use the side data from AVStream.codecpar

[Andreas Rheinhardt]

James Almer:
> Deprecate AVStream.side_data and its helpers in favor of the AVStream's
> codecpar.side_data.
> 
> This will considerably simplify the propagation of global side data to decoders
> and from encoders. Instead of having to do it inside packets, it will be
> available during init().
> Global and frame specific side data will therefore be distinct.
> 
> Signed-off-by: James Almer <jamrial at gmail.com>
> ---

...

> diff --git a/libavformat/avformat.h b/libavformat/avformat.h
> index a8e245000f..8a3fe137fa 100644
> --- a/libavformat/avformat.h
> +++ b/libavformat/avformat.h
> @@ -940,6 +940,7 @@ typedef struct AVStream {
>       */
>      AVPacket attached_pic;
>  
> +#if FF_API_AVSTREAM_SIDE_DATA
>      /**
>       * An array of side data that applies to the whole stream (i.e. the
>       * container does not allow it to change between packets).
> @@ -956,13 +957,20 @@ typedef struct AVStream {
>       *
>       * Freed by libavformat in avformat_free_context().
>       *
> -     * @see av_format_inject_global_side_data()
> +     * @deprecated use AVStream's @ref AVCodecParameters.side_data
> +     *             "codecpar side data".
>       */
> +    attribute_deprecated
>      AVPacketSideData *side_data;
>      /**
>       * The number of elements in the AVStream.side_data array.
> +     *
> +     * @deprecated use AVStream's @ref AVCodecParameters.side_data
> +     *             "codecpar side data".
>       */
> +    attribute_deprecated
>      int            nb_side_data;
> +#endif
>  
>      /**
>       * Flags indicating events happening on the stream, a combination of
> @@ -1720,11 +1728,18 @@ typedef struct AVFormatContext {
>      int (*io_close2)(struct AVFormatContext *s, AVIOContext *pb);
>  } AVFormatContext;
>  
> +#if FF_API_AVSTREAM_SIDE_DATA
>  /**
>   * This function will cause global side data to be injected in the next packet
>   * of each stream as well as after any subsequent seek.
> + *
> + * @deprecated global side data is always available in every AVStream's
> + *             @ref AVCodecParameters.side_data "codecpar side data" array.
> + * @see av_packet_side_data_get()
>   */
> +attribute_deprecated
>  void av_format_inject_global_side_data(AVFormatContext *s);
> +#endif

This is not a "helper" of AVStream.side_data at all. Whereas porting
code to the new API is straightforward for the other deprecated
functions, this one is not. In fact, removing it adds complications for
users, because a user could simply inject global side data via this
function and then only look at packets, the user now has to add special
code to also inspect the global side data.

One example affected by this is my current patch for the Matroska
demuxer to output the palette as stream side data and not as packet side
data as the other demuxers do. Without
av_format_inject_global_side_data(), this will break every user
searching for palette data only in packet side data and not in stream
side data. In fact, one such user is libavformat itself, because all
muxers (nut and avi and mov (the latter two via rawutils)) only search
for this type of side data in packets.

As you can see in that patch, it fixes an issue with seeking where the
palette is attached to a packet read during avformat_find_stream_info()
that is discarded afterwards due to a seek. The same issue also exists
in avi, yet the solution is not so simple there because avi has a second
way to export palettes and due to the requirement that global and
packet-specific side data is supposed to be distinct one can't simply
attach the palette extracted from extradata as stream side-data. Which
makes me think that this requirement is not really reasonable.

- Andreas