[FFmpeg-devel] libavutil/imgutils: UBSan nullptr-with-offset in av_image_fill_pointer

[Brian Kim]

I'm using clang 10.0.0 and configuring with with `configure --enable-gpl
--enable-libass --enable-libfdk-aac --enable-libmp3lame
--enable-libopencore-amrnb --enable-libopencore-amrwb --enable-librtmp
--enable-libtheora --enable-libvorbis --enable-libopus --enable-libx264
--enable-libvpx --enable-nonfree --enable-version3 --disable-optimizations
--disable-stripping --enable-debug=3 --toolchain=clang-usan` (plus
`--extra-cflags=-fno-sanitize-recover=pointer-overflow
--extra-ldflags=-fno-sanitize-recover=pointer-overflow` to crash when we
run into the UB)

When I run `make fate -j 12 SAMPLES=fate-suite`, several tests (e.g.
api-mjpeg-codec-param) fail with something like the following in the error
logs:

libavutil/imgutils.c:139:29: runtime error: applying non-zero offset 131072
to null pointer
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior
libavutil/imgutils.c:139:29 in

 It looks like this check was added in clang 10 (
https://reviews.llvm.org/D67122)

On Thu, Jul 2, 2020 at 8:06 AM Michael Niedermayer <michael at niedermayer.cc>
wrote:

> On Wed, Jul 01, 2020 at 11:14:13AM -0700, Brian Kim wrote:
> > While running under Clang's UndefinedBehaviorSanitizer, I found a few
> > places where av_image_fill_pointers is called before buffers for the
> image
> > are allocated, so ptr is passed in as NULL.
> >
> > This leads to (currently harmless) UB when the plane sizes are added to
> the
> > null pointer, so I was wondering if there was interest in avoiding it?
> >
> > I've attached a patch to expose some extra utilities and avoid passing in
> > the null pointer, but is this an appropriate way to work around it?
>
> how can these cases be reproduced ?
>
> thx
>
> [...]
> --
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> The smallest minority on earth is the individual. Those who deny
> individual rights cannot claim to be defenders of minorities. - Ayn Rand
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".