[FFmpeg-devel] [PATCH v2 11/12] lavc/vaapi_decode: use dynamic frame pool for output frames with libva2
Xiang, Haihao
haihao.xiang at intel.com
Thu Feb 8 06:15:22 EET 2024
On Wo, 2024-01-31 at 02:26 +0000, Xiang, Haihao wrote:
> On Di, 2024-01-30 at 19:07 +0000, Mark Thompson wrote:
> > On 30/01/2024 06:30, Xiang, Haihao wrote:
> > > On Ma, 2024-01-29 at 21:58 +0000, Mark Thompson wrote:
> > > > On 26/01/2024 07:25, Xiang, Haihao wrote:
> > > > > On Wo, 2023-12-20 at 15:10 +0800, Xiang, Haihao wrote:
> > > > > > From: Haihao Xiang <haihao.xiang at intel.com>
> > > > > >
> > > > > > This allows a downstream element stores more frames from VAAPI
> > > > > > decoders and fixes error in get_buffer()
> > > > > >
> > > > > > $ ffmpeg -hwaccel vaapi -hwaccel_output_format vaapi -i
> > > > > > input_100frames.mp4 \
> > > > > > -vf reverse -an -f null -
> > > > > > ...
> > > > > > [h264 @ 0x557a075a1400] get_buffer() failed
> > > > > > [h264 @ 0x557a075a1400] thread_get_buffer() failed
> > > > > > [h264 @ 0x557a075a1400] decode_slice_header error
> > > > > > [h264 @ 0x557a075a1400] no frame!
> > > > > >
> > > > > > Signed-off-by: Haihao Xiang <haihao.xiang at intel.com>
> > > > > > ---
> > > > > > libavcodec/vaapi_decode.c | 36 ++++++++++++++++++++--------------
> > > > > > --
> > > > > > 1 file changed, 20 insertions(+), 16 deletions(-)
> > > > > >
> > > > > > diff --git a/libavcodec/vaapi_decode.c b/libavcodec/vaapi_decode.c
> > > > > > index ceac769c52..8cc29e96f9 100644
> > > > > > --- a/libavcodec/vaapi_decode.c
> > > > > > +++ b/libavcodec/vaapi_decode.c
> > > > > > @@ -601,22 +601,26 @@ static int
> > > > > > vaapi_decode_make_config(AVCodecContext
> > > > > > *avctx,
> > > > > > if (err < 0)
> > > > > > goto fail;
> > > > > >
> > > > > > - frames->initial_pool_size = 1;
> > > > > > - // Add per-codec number of surfaces used for storing
> > > > > > reference
> > > > > > frames.
> > > > > > - switch (avctx->codec_id) {
> > > > > > - case AV_CODEC_ID_H264:
> > > > > > - case AV_CODEC_ID_HEVC:
> > > > > > - case AV_CODEC_ID_AV1:
> > > > > > - frames->initial_pool_size += 16;
> > > > > > - break;
> > > > > > - case AV_CODEC_ID_VP9:
> > > > > > - frames->initial_pool_size += 8;
> > > > > > - break;
> > > > > > - case AV_CODEC_ID_VP8:
> > > > > > - frames->initial_pool_size += 3;
> > > > > > - break;
> > > > > > - default:
> > > > > > - frames->initial_pool_size += 2;
> > > > > > + if (CONFIG_VAAPI_1)
> > > > > > + frames->initial_pool_size = 0;
> > > > > > + else {
> > > > > > + frames->initial_pool_size = 1;
> > > > > > + // Add per-codec number of surfaces used for storing
> > > > > > reference
> > > > > > frames.
> > > > > > + switch (avctx->codec_id) {
> > > > > > + case AV_CODEC_ID_H264:
> > > > > > + case AV_CODEC_ID_HEVC:
> > > > > > + case AV_CODEC_ID_AV1:
> > > > > > + frames->initial_pool_size += 16;
> > > > > > + break;
> > > > > > + case AV_CODEC_ID_VP9:
> > > > > > + frames->initial_pool_size += 8;
> > > > > > + break;
> > > > > > + case AV_CODEC_ID_VP8:
> > > > > > + frames->initial_pool_size += 3;
> > > > > > + break;
> > > > > > + default:
> > > > > > + frames->initial_pool_size += 2;
> > > > > > + }
> > > > > > }
> > > > > > }
> > > > > >
> > > > >
> > > > > Hi Mark,
> > > > >
> > > > > Do you have any comment about dynamic frame pool used in vaapi ?
> > > >
> > > > Are we completely sure that there are no driver/hardware combinations
> > > > which
> > > > rely on this still used?
> > >
> > > I tested this patch with i965, iHD and radeonsi drivers on Linux and
> > > vaon12
> > > driver on Windows. But honestly I am not sure whether there is a driver
> > > which
> > > works with fixed frame pool only.
> >
> > How does the vaon12 driver work with this given that some D3D12 devices
> > require a fixed array texture?
>
> Honestly I don't know. I don't have such HWs for testing.
>
> >
> > Note that the interesting test here is not the most recent version of any of
> > these things. Rather, it is the older versions which exist in a
> > distribution
> > configuration which we still want to support, for example Ubuntu 20.04.
>
> This patch is based on libva2, FFmpeg still works with libva. For older
> versions
> , user may use libva.
>
> >
> > I'm also unclear to what degree this might depend on the hardware being
> > used.
> > Certainly in D3D12 whether the fixed array texture is required depends on
> > the
> > actual hardware support.
>
> How about add a quirk for workable driver(s) only ? We won't be concerned by
> other drivers.
>
> diff --git a/libavutil/hwcontext_vaapi.c b/libavutil/hwcontext_vaapi.c
> index e6f45f8fde..aecdd55728 100644
> --- a/libavutil/hwcontext_vaapi.c
> +++ b/libavutil/hwcontext_vaapi.c
> @@ -374,7 +374,7 @@ static const struct {
> {
> "Intel iHD",
> "ubit",
> - AV_VAAPI_DRIVER_QUIRK_ATTRIB_MEMTYPE,
> + AV_VAAPI_DRIVER_QUIRK_ATTRIB_MEMTYPE |
> AV_VAAPI_DRIVER_QUIRK_DYNAMIC_SURFACE_POOL,
> },
> {
> "VDPAU wrapper",
> diff --git a/libavutil/hwcontext_vaapi.h b/libavutil/hwcontext_vaapi.h
> index 259c6f5dbd..c4d2709224 100644
> --- a/libavutil/hwcontext_vaapi.h
> +++ b/libavutil/hwcontext_vaapi.h
> @@ -60,6 +60,12 @@ enum {
> * and the results of the vaQuerySurfaceAttributes() call will be faked.
> */
> AV_VAAPI_DRIVER_QUIRK_SURFACE_ATTRIBUTES = (1 << 3),
> +
> + /**
> + * The driver (and the underlying HW) supports dynamic surface pool.
> + * The vaCreateContext() call doesn't require a fixed array surfaces.
> + */
> + AV_VAAPI_DRIVER_QUIRK_DYNAMIC_SURFACE_POOL = (1 << 4),
> };
>
> /**
Hi Mark,
Is there any comment or concern about adding a quirk for workable drivers ? We
may use a dynamic frame pool in vaapi decoders and filters for workable drivers
only.
Note since commit e0da916b, a command below doesn't work with the current fixed
frame pool used in vaapi filters.
$ ffmpeg -hwaccel_output_format vaapi -hwaccel vaapi -i input.mp4 -vf
'scale_vaapi=w=720:h=480' -c:v hevc_vaapi -f null -
[...]
[vf#0:0 @ 0x562847b01050] Error while filtering: Cannot allocate memory
[vf#0:0 @ 0x562847b01050] Task finished with error code: -12 (Cannot allocate
memory)
[vf#0:0 @ 0x562847b01050] Terminating thread with return code -12 (Cannot
allocate memory)
[...]
Thanks
Haihao
More information about the ffmpeg-devel
mailing list