[FFmpeg-devel] [PATCH] avcodec/hevc_ps: fix the problem of memcmp losing effectiveness

[Andreas Rheinhardt]

Mark Thompson:
> On 28/03/2024 13:15, tong1.wu-at-intel.com at ffmpeg.org wrote:
>> From: Tong Wu <tong1.wu at intel.com>
>>
>> HEVCHdrParams* receives a pointer which points to a dynamically
>> allocated memory block. It causes the memcmp always returning 1.
>> Add a function to do the comparision. A condition is also added to
>> avoid malloc(0).
>>
>> Signed-off-by: Tong Wu <tong1.wu at intel.com>
>> ---
>>   libavcodec/hevc_ps.c | 20 ++++++++++++++++----
>>   libavcodec/hevc_ps.h |  4 +++-
>>   2 files changed, 19 insertions(+), 5 deletions(-)
> 
> It doesn't seem like this method works at all, even before the recent
> change with the pointer.
> 
> Structs can contain arbitrary padding, and any write to the struct makes
> the padding unspecified.  memcmp() is therefore never valid as a method
> of comparing after writing some fields, as done here.  (It could only be
> valid if the structs compared were made by memcpy() with no fields
> written directly.)
> 
> The problem is mostly harmless because the nondeterministic replacement
> of structs which we were expecting to be equivalent doesn't actually
> change anything, so why don't we just remove the comparison and always
> replace?
> 

remove_vps() also removes any SPS referencing this VPS (and remove_sps()
does the same with PPS). Therefore if you simply repeat a VPS without
also repeating the other parameter sets directly after the new VPS and
before the first video NALU after the VPS, your extradata will have been
discarded.
This is not what the spec says.

- Andreas