[FFmpeg-cvslog] avcodec/asvenc: Fix crash with unaligned pointers/linesizes

[Andreas Rheinhardt]

ffmpeg | branch: master | Andreas Rheinhardt <andreas.rheinhardt at outlook.com> | Tue May 27 15:38:17 2025 +0200| [75960ac2708659344bc33b4c108e4a49a0d3184e] | committer: Andreas Rheinhardt

avcodec/asvenc: Fix crash with unaligned pointers/linesizes

This happens on systems where get_pixels really needs
to be properly aligned, like ARMV7 or RISC-V. For these
systems, 0401ca714a2714743573e27c384ffa810fd31a92 caused
a bus error for the vsynth3-asv[12] tests, because
the stride in these tests is unaligned. See e.g.
https://fate.ffmpeg.org/report.cgi?slot=armv7-linux-gcc-13&time=20250527020548
https://fate.ffmpeg.org/report.cgi?slot=rv64gcvb-linux-gnu-gcc&time=20250527001827

It can also happen (even before said commit) if the pointers
itself are unaligned, e.g. by using the crop filter:
ffmpeg -filter_complex nullsrc=s=740x576:r=25,format=yuv420p,crop=w=720:x=2 \
-c:v asv2 -f null -

The alignment requirements for the frames passed to encoders are
mostly undocumented; the only thing I could find is the documentation
of AVFrame.linesize: "For video the linesizes should be multiples
of the CPUs alignment preference". This means that the FFmpeg cli
violates our API.

Yet as the above command line shows, it can also happen with
unaligned pointers and there does not seem to be a prohibition
of this, so we need to handle this case. This commit does so
by using get_pixels_unaligned when needed.

Reviewed-by: Martin Storsjö <martin at martin.st>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt at outlook.com>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=75960ac2708659344bc33b4c108e4a49a0d3184e
---

 libavcodec/asvenc.c | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)

diff --git a/libavcodec/asvenc.c b/libavcodec/asvenc.c
index bcdb5cfbe2..ba48aa8f08 100644
--- a/libavcodec/asvenc.c
+++ b/libavcodec/asvenc.c
@@ -45,6 +45,10 @@ typedef struct ASVEncContext {
 
     PutBitContext pb;
 
+    void (*get_pixels)(int16_t *restrict block,
+                       const uint8_t *pixels,
+                       ptrdiff_t stride);
+
     PixblockDSPContext pdsp;
     FDCTDSPContext fdsp;
     DECLARE_ALIGNED(32, int16_t, block)[6][64];
@@ -219,16 +223,16 @@ static inline void dct_get(ASVEncContext *a, const AVFrame *frame,
     const uint8_t *ptr_cb = frame->data[1] + (mb_y *  8 * frame->linesize[1]) + mb_x *  8;
     const uint8_t *ptr_cr = frame->data[2] + (mb_y *  8 * frame->linesize[2]) + mb_x *  8;
 
-    a->pdsp.get_pixels(block[0], ptr_y,                    linesize);
-    a->pdsp.get_pixels(block[1], ptr_y + 8,                linesize);
-    a->pdsp.get_pixels(block[2], ptr_y + 8 * linesize,     linesize);
-    a->pdsp.get_pixels(block[3], ptr_y + 8 * linesize + 8, linesize);
+    a->get_pixels(block[0], ptr_y,                    linesize);
+    a->get_pixels(block[1], ptr_y + 8,                linesize);
+    a->get_pixels(block[2], ptr_y + 8 * linesize,     linesize);
+    a->get_pixels(block[3], ptr_y + 8 * linesize + 8, linesize);
     for (i = 0; i < 4; i++)
         a->fdsp.fdct(block[i]);
 
     if (!(a->c.avctx->flags & AV_CODEC_FLAG_GRAY)) {
-        a->pdsp.get_pixels(block[4], ptr_cb, frame->linesize[1]);
-        a->pdsp.get_pixels(block[5], ptr_cr, frame->linesize[2]);
+        a->get_pixels(block[4], ptr_cb, frame->linesize[1]);
+        a->get_pixels(block[5], ptr_cr, frame->linesize[2]);
         for (i = 4; i < 6; i++)
             a->fdsp.fdct(block[i]);
     }
@@ -297,6 +301,13 @@ static int encode_frame(AVCodecContext *avctx, AVPacket *pkt,
     if (ret < 0)
         return ret;
 
+    if ((uintptr_t)pict->data[0] & 7 || pict->linesize[0] & 7 ||
+        (uintptr_t)pict->data[1] & 7 || pict->linesize[1] & 7 ||
+        (uintptr_t)pict->data[2] & 7 || pict->linesize[2] & 7)
+        a->get_pixels = a->pdsp.get_pixels_unaligned;
+    else
+        a->get_pixels = a->pdsp.get_pixels;
+
     init_put_bits(&a->pb, pkt->data, pkt->size);
 
     for (int mb_y = 0; mb_y < c->mb_height2; mb_y++) {