[FFmpeg-devel] [PATCH 03/11] avformat/avformat: use the side data from AVStream.codecpar

[James Almer]

On 10/6/2023 12:04 AM, Andreas Rheinhardt wrote:
> James Almer:
>> Deprecate AVStream.side_data and its helpers in favor of the AVStream's
>> codecpar.side_data.
>>
>> This will considerably simplify the propagation of global side data to decoders
>> and from encoders. Instead of having to do it inside packets, it will be
>> available during init().
>> Global and frame specific side data will therefore be distinct.
>>
>> Signed-off-by: James Almer <jamrial at gmail.com>
>> ---
> 
> ...
> 
>> diff --git a/libavformat/avformat.h b/libavformat/avformat.h
>> index a8e245000f..8a3fe137fa 100644
>> --- a/libavformat/avformat.h
>> +++ b/libavformat/avformat.h
>> @@ -940,6 +940,7 @@ typedef struct AVStream {
>>        */
>>       AVPacket attached_pic;
>>   
>> +#if FF_API_AVSTREAM_SIDE_DATA
>>       /**
>>        * An array of side data that applies to the whole stream (i.e. the
>>        * container does not allow it to change between packets).
>> @@ -956,13 +957,20 @@ typedef struct AVStream {
>>        *
>>        * Freed by libavformat in avformat_free_context().
>>        *
>> -     * @see av_format_inject_global_side_data()
>> +     * @deprecated use AVStream's @ref AVCodecParameters.side_data
>> +     *             "codecpar side data".
>>        */
>> +    attribute_deprecated
>>       AVPacketSideData *side_data;
>>       /**
>>        * The number of elements in the AVStream.side_data array.
>> +     *
>> +     * @deprecated use AVStream's @ref AVCodecParameters.side_data
>> +     *             "codecpar side data".
>>        */
>> +    attribute_deprecated
>>       int            nb_side_data;
>> +#endif
>>   
>>       /**
>>        * Flags indicating events happening on the stream, a combination of
>> @@ -1720,11 +1728,18 @@ typedef struct AVFormatContext {
>>       int (*io_close2)(struct AVFormatContext *s, AVIOContext *pb);
>>   } AVFormatContext;
>>   
>> +#if FF_API_AVSTREAM_SIDE_DATA
>>   /**
>>    * This function will cause global side data to be injected in the next packet
>>    * of each stream as well as after any subsequent seek.
>> + *
>> + * @deprecated global side data is always available in every AVStream's
>> + *             @ref AVCodecParameters.side_data "codecpar side data" array.
>> + * @see av_packet_side_data_get()
>>    */
>> +attribute_deprecated
>>   void av_format_inject_global_side_data(AVFormatContext *s);
>> +#endif
> 
> This is not a "helper" of AVStream.side_data at all. Whereas porting
> code to the new API is straightforward for the other deprecated
> functions, this one is not. In fact, removing it adds complications for
> users, because a user could simply inject global side data via this
> function and then only look at packets, the user now has to add special
> code to also inspect the global side data.
> 
> One example affected by this is my current patch for the Matroska
> demuxer to output the palette as stream side data and not as packet side
> data as the other demuxers do. Without
> av_format_inject_global_side_data(), this will break every user
> searching for palette data only in packet side data and not in stream
> side data. In fact, one such user is libavformat itself, because all
> muxers (nut and avi and mov (the latter two via rawutils)) only search
> for this type of side data in packets.
> 
> As you can see in that patch, it fixes an issue with seeking where the
> palette is attached to a packet read during avformat_find_stream_info()
> that is discarded afterwards due to a seek. The same issue also exists
> in avi, yet the solution is not so simple there because avi has a second
> way to export palettes and due to the requirement that global and
> packet-specific side data is supposed to be distinct one can't simply
> attach the palette extracted from extradata as stream side-data. Which
> makes me think that this requirement is not really reasonable.
> 
> - Andreas

I can remove the deprecation from this patch. No strong feelings about 
it, and it can be done later if needed.