[FFmpeg-devel] [PATCH v3 1/1] avfilter/buffersink: Add video frame allocation callback

[John Cox]

Hi

>On 7/22/2023 2:03 PM, John Cox wrote:
>> Add a callback to enable user allocation of video frames on the final
>> stage of a filter chain.
>> 
>> Signed-off-by: John Cox <jc at kynesim.co.uk>
>> ---
>>   libavfilter/buffersink.c | 21 +++++++++++++++++++++
>>   libavfilter/buffersink.h | 27 +++++++++++++++++++++++++++
>>   libavfilter/version.h    |  2 +-
>>   3 files changed, 49 insertions(+), 1 deletion(-)
>> 
>> diff --git a/libavfilter/buffersink.c b/libavfilter/buffersink.c
>> index 306c283f77..e99d444530 100644
>> --- a/libavfilter/buffersink.c
>> +++ b/libavfilter/buffersink.c
>> @@ -62,6 +62,11 @@ typedef struct BufferSinkContext {
>>       int sample_rates_size;
>>   
>>       AVFrame *peeked_frame;
>> +
>> +    union {
>> +        AVBuffersinkAllocVideoFrameFunc video;
>> +    } alloc_cb;
>> +    void * alloc_v;
>>   } BufferSinkContext;
>>   
>>   #define NB_ITEMS(list) (list ## _size / sizeof(*list))
>> @@ -154,6 +159,21 @@ int attribute_align_arg av_buffersink_get_samples(AVFilterContext *ctx,
>>       return get_frame_internal(ctx, frame, 0, nb_samples);
>>   }
>>   
>> +static AVFrame *alloc_video_buffer(AVFilterLink *link, int w, int h)
>> +{
>> +    AVFilterContext *const ctx = link->dst;
>> +    BufferSinkContext *const buf = ctx->priv;
>> +    return buf->alloc_cb.video ? buf->alloc_cb.video(ctx, buf->alloc_v, w, h) :
>> +                                 ff_default_get_video_buffer(link, w, h);
>
>Does the AVBuffersinkAllocVideoFrameFunc function have access to the 
>AVFilterLink ff_default_get_video_buffer() gets? I assume it'd be needed 
>to get values like pixel format. If so, it should be documented how, but 
>maybe it's easier to just pass link here instead of the AVFilterContext, 
>and let the caller access link->dst if needed.

Unless I'm misreading stuff you can get at AVFilterLink from from
AVFilterContext so I think AVFilterContext is good.
If you are suggesting that the documentation of AVFilterContext & Link
could be improved I feel that is outside the scope of this patch.

>Also, looking at ff_default_get_video_buffer(), a hardware enabled 
>filter may populate hw_frames_ctx, and that's a field only available in 
>the AVFilterLink and that should not be accessed from outside lavfi, 
>which includes a caller defined function used for this callback.

I'm afraid I don't quite understand what you are suggesting here. What
would you have me do differently?

Regards

JC

>> +}
>> +
>> +void av_buffersink_set_alloc_video_frame(AVFilterContext *ctx, AVBuffersinkAllocVideoFrameFunc cb, void *v)
>> +{
>> +    BufferSinkContext *const buf = ctx->priv;
>> +    buf->alloc_cb.video = cb;
>> +    buf->alloc_v = v;
>> +}
>> +
>>   static av_cold int common_init(AVFilterContext *ctx)
>>   {
>>       BufferSinkContext *buf = ctx->priv;
>> @@ -381,6 +401,7 @@ static const AVFilterPad avfilter_vsink_buffer_inputs[] = {
>>       {
>>           .name = "default",
>>           .type = AVMEDIA_TYPE_VIDEO,
>> +        .get_buffer = {.video = alloc_video_buffer},
>>       },
>>   };
>>   
>> diff --git a/libavfilter/buffersink.h b/libavfilter/buffersink.h
>> index 64e08de53e..2419d1bd80 100644
>> --- a/libavfilter/buffersink.h
>> +++ b/libavfilter/buffersink.h
>> @@ -166,6 +166,33 @@ int av_buffersink_get_frame(AVFilterContext *ctx, AVFrame *frame);
>>    */
>>   int av_buffersink_get_samples(AVFilterContext *ctx, AVFrame *frame, int nb_samples);
>>   
>> +/**
>> + * Callback from av_buffersink_set_alloc_video_frame to allocate
>> + * a frame
>> + *
>> + * @param ctx pointer to a context of the abuffersink AVFilter.
>> + * @param v opaque pointer passed to
>> + *          av_buffersink_set_alloc_video_frame
>> + * @param w width of frame to allocate
>> + * @param height of frame to allocate
>> + *
>> + * @return
>> + *         - The newly allocated frame
>> + *         - NULL if error
>> + */
>> +typedef AVFrame *(*AVBuffersinkAllocVideoFrameFunc)(AVFilterContext * ctx, void * v, int w, int h);
>> +
>> +/**
>> + * Set a video frame allocation method for buffersink
>> + *
>> + * @param ctx pointer to a context of the abuffersink AVFilter.
>> + * @param cb Callback to the allocation function. If set to NULL
>> + *           then the default avfilter allocation function will
>> + *           be used.
>> + * @param v  Opaque to pass to the allocation function
>> + */
>> +void av_buffersink_set_alloc_video_frame(AVFilterContext *ctx, AVBuffersinkAllocVideoFrameFunc cb, void * v);
>> +
>>   /**
>>    * @}
>>    */
>> diff --git a/libavfilter/version.h b/libavfilter/version.h
>> index c001693e3c..54950497be 100644
>> --- a/libavfilter/version.h
>> +++ b/libavfilter/version.h
>> @@ -31,7 +31,7 @@
>>   
>>   #include "version_major.h"
>>   
>> -#define LIBAVFILTER_VERSION_MINOR   8
>> +#define LIBAVFILTER_VERSION_MINOR   9
>>   #define LIBAVFILTER_VERSION_MICRO 102
>>   
>>   
>_______________________________________________
>ffmpeg-devel mailing list
>ffmpeg-devel at ffmpeg.org
>https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>To unsubscribe, visit link above, or email
>ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".