[FFmpeg-devel] [PATCH] Fix crash if av_vdpau_bind_context() is not used.

[James Almer]

On 10/9/2017 8:30 AM, wm4 wrote:
> On Mon, 9 Oct 2017 03:04:53 +0300
> Ivan Kalvachev <ikalvachev at gmail.com> wrote:
> 
>> The public functions av_alloc_vdpaucontext() and
>> av_vdpau_alloc_context() are allocating AVVDPAUContext
>> structure that is supposed to be placed in avctx->hwaccel_context.
>>
>> However the rest of libavcodec/vdpau.c uses avctx->hwaccel_context
>> as struct VDPAUHWContext, that is bigger and does contain
>> AVVDPAUContext as first member.
>>
>> The usage includes write to the new variables in the bigger stuct,
>> without checking for block size.
>>
>> Fix by always allocating the bigger structure.
>>
>> BTW,
>> I have no idea why the new fields haven't simply been added to the
>> existing struct...
>> It seems that the programmer who wrote this has been aware of the problem,
>> because av_vdpau_bind_context reallocates the structure.
>>
>> It might be good idea to check the other usages of this reallocation function.
>>
>> Best Regards
>>    Ivan Kalvachev
> 
> IMO not really worth fixing at this point, because this is the old-old
> vdpau API. Even av_vdpau_bind_context() (which does not require using
> av_alloc_vdpaucontext()) is deprecated. Or rather should be - I just
> haven't bothered deprecating it because the deprecation dance is too
> messy. In any case, you shouldn't use any of those APIs - use the
> generic hwaccel API instead (setting hw_frames_ctx or hw_device_ctx).

av_vdpau_alloc_context() is not deprecated, neither here on in libav.
APIChanges only has a mention that av_vdpau_bind_context() should be
used instead and that's it. It doesn't even seem to mention that neither
should be used at all in favor of the hwaccel API.

If all these are truly deprecated, then I'd recommend doing the dance
asap to the clock may start ticking.