[FFmpeg-devel] [PATCH v4] avcodec/decode: validate hw_frames_ctx when AVHWAccel.free_frame_priv is used

[Rogozhkin, Dmitry V]

On Tue, 2023-11-21 at 06:17 +0100, Lynne wrote:
> Nov 21, 2023, 05:38 by dmitry.v.rogozhkin-at-intel.com at ffmpeg.org:
> 
> > Validate that a hw_frames_ctx is available before using it for
> > the AVHWAccel.free_frame_priv callback, and don't require it to
> > be present when the callback is not in use by the HWAccel.
> > 
> > v2: check for free_frame_priv (Hendrik)
> > v3: return EINVAL (Christoph Reiter)
> > v4: better commit message (Hendrik)
> > 
> > See[1]: https://github.com/msys2/MINGW-packages/pull/19050
> > Fixes: be07145109 ("avcodec: add AVHWAccel.free_frame_priv
> > callback")
> > CC: Lynne <dev at lynne.ee>
> > CC: Christoph Reiter <reiter.christoph at gmail.com>
> > Signed-off-by: Dmitry Rogozhkin <dmitry.v.rogozhkin at intel.com>
> > ---
> >  libavcodec/decode.c | 18 +++++++++++++-----
> >  1 file changed, 13 insertions(+), 5 deletions(-)
> > 
> > diff --git a/libavcodec/decode.c b/libavcodec/decode.c
> > index ad39021..50c3995 100644
> > --- a/libavcodec/decode.c
> > +++ b/libavcodec/decode.c
> > @@ -1838,17 +1838,25 @@ int ff_copy_palette(void *dst, const
> > AVPacket *src, void *logctx)
> >  int ff_hwaccel_frame_priv_alloc(AVCodecContext *avctx, void
> > **hwaccel_picture_private)
> >  {
> >  const FFHWAccel *hwaccel = ffhwaccel(avctx->hwaccel);
> > -    AVHWFramesContext *frames_ctx;
> >  
> >  if (!hwaccel || !hwaccel->frame_priv_data_size)
> >  return 0;
> >  
> >  av_assert0(!*hwaccel_picture_private);
> >  
> > -    frames_ctx = (AVHWFramesContext *)avctx->hw_frames_ctx->data;
> > -    *hwaccel_picture_private = ff_refstruct_alloc_ext(hwaccel-
> > >frame_priv_data_size, 0,
> > -                                                      frames_ctx-
> > >device_ctx,
> > -                                                      hwaccel-
> > >free_frame_priv);
> > +    if (hwaccel->free_frame_priv) {
> > +        AVHWFramesContext *frames_ctx;
> > +
> > +        if (!avctx->hw_frames_ctx)
> > +            return AVERROR(EINVAL);
> > +
> > +        *hwaccel_picture_private = ff_refstruct_alloc_ext(hwaccel-
> > >frame_priv_data_size, 0,
> > +                                                          frames_c
> > tx->device_ctx,
> > +                                                          hwaccel-
> > >free_frame_priv);
> > +    } else {
> > +        *hwaccel_picture_private = ff_refstruct_allocz(hwaccel-
> > >frame_priv_data_size);
> > +    }
> > +
> >  if (!*hwaccel_picture_private)
> >  return AVERROR(ENOMEM);
> > 
> 
> You never set frames_ctx in this patch.

Thank you for the catch. Yep, I did a type doing v3. Fixed now. See v5.

> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".