[FFmpeg-devel] [PATCH V4 2/4] libavfilter/buffersink.c: unref private_ref when frame leaves libavfilter

[Guo, Yejun]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of Nicolas
> George
> Sent: 2021年3月1日 23:07
> To: FFmpeg development discussions and patches <ffmpeg-devel at ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH V4 2/4] libavfilter/buffersink.c: unref
> private_ref when frame leaves libavfilter
> 
> Guo, Yejun (12021-03-01):
> > Actually, I think private_ref in libavfilter can only be used for an
> > exclusive usage at a time.
> 
> Exactly. If we use it for this, then we cannot use for anything else in libavfilter.
> This use seems too specific to warrant dedicating such an unique field to it,
> even though we do not have a better use in sight.
> 
> > As Paul mentioned, I think AVFrame.metadata is a better choice.
> 
> If you can express it as a string or set of strings with a clear syntax that can
> easily be parsed, then possibly, yes.

ooo, it is not easy to express the bounding boxes as strings in AVDictionaryEntry.value,
the bounding box has several data members, and they are data and have high possibility
to contain '\0' in the middle of the data. So, we might not use AVFrame.metadata.

So, where to put the bounding boxes (object detection result generated by vf_dnn_detect),
I now see several possible methods which all have positive/negative comments:

1. Add into side data
The final result is to be in side data since it might be used by new encoders in the future, 
but this method changes the API.

1.1 We just add a new enum for side data, and keep the .h file (for structs) internal at first.
There's comment that this is not allowed. (I personally prefer this one.)

1.2 We add enum for side data, and also export the .h file as part of FFmpeg API.
The risk is that we might change the structs in .h file later, it breaks API.
We need a versioning management for the struct, just like film grain as explained
at http://ffmpeg.org/pipermail/ffmpeg-devel/2021-February/276586.html .

2. Use private_ref
Use private_ref for bounding boxes at first, and then change to side data when it is required.
The disadvantage is that during the period, we cannot use it for anything else in libavfilter.

any comment or any other suggestion? thanks.