[FFmpeg-devel] [PATCH v4] Unbreak av_malloc_max(0) API/ABI

[Joakim Tjernlund]

Yet a ping ...

This is a simple technical patch, it just needs a policy decision.
Can I have one ?

   Jocke

On Thu, 2020-10-22 at 14:17 +0200, Joakim Tjernlund wrote:
> Ping ?
> 
>  Jocke
> 
> On Fri, 2020-10-16 at 10:57 +0200, Joakim Tjernlund wrote:
> > From https://bugs.chromium.org/p/chromium/issues/detail?id=1095962
> > ----------------------------
> > This seems to be caused by the custom handling of "av_max_alloc(0)" in
> > Chromium's ffmpeg fork to mean unlimited (added in [1]).
> > 
> > Upstream ffmpeg doesn't treat 0 as a special value; versions before 4.3 seemingly worked
> > because 32 was subtracted from max_alloc_size (set to 0 by Chromium) resulting in an
> > integer underflow, making the effective limit be SIZE_MAX - 31.
> > 
> > Now that the above underflow doesn't happen, the tab just crashes. The upstream change
> > for no longer subtracting 32 from max_alloc_size was included in ffmpeg 4.3. [2]
> > 
> > [1] https://chromium-review.googlesource.com/c/chromium/third_party/ffmpeg/+/73563
> > [2] https://github.com/FFmpeg/FFmpeg/commit/731c77589841
> > ---------------------------
> > 
> > Restore av_malloc_max(0) to MAX_INT fixing MS Teams, Discord older chromium etc.
> > 
> > Signed-off-by: Joakim Tjernlund <joakim.tjernlund at infinera.com>
> > ---
> > 
> >  v2: Cover the full API range 0-31
> > 
> >  v3: Closer compat with < 4.3 ffmpeg
> > 
> >  v4: Adjust size accoriding to Andreas Rheinhardt comments
> > 
> >  libavutil/mem.c | 2 ++
> >  1 file changed, 2 insertions(+)
> > 
> > diff --git a/libavutil/mem.c b/libavutil/mem.c
> > index cfb6d8a..44870a9 100644
> > --- a/libavutil/mem.c
> > +++ b/libavutil/mem.c
> > @@ -71,6 +71,8 @@ void  free(void *ptr);
> >  static size_t max_alloc_size= INT_MAX;
> >  
> > 
> >  void av_max_alloc(size_t max){
> > +    if (max < 32)
> > +        max = SIZE_MAX - 32 + max; /* be compatible to older(< 4.3) versions */
> >      max_alloc_size = max;
> >  }
> >  
> > 
>