[FFmpeg-devel] [PATCH 1/5] avcodec/rl2: Remove wrong check
Andreas Rheinhardt
andreas.rheinhardt at outlook.com
Sun Oct 2 04:23:18 EEST 2022
Andreas Rheinhardt:
> This check is intended to be avoid buffer overflows,
> yet there are four problems with it:
> 1. It has an in-built off-by-one error: len == out_end - out
> is perfectly fine and nothing to worry about.
> This off-by-one error led to the pixel in the lower-right corner
> not being set properly for the back frame of the sample from
> the rl2 FATE-test. This pixel is copied to every frame which
> is the reason for the update to the reference file of said test.
> With this patch, the output of the decoder matches the output
> as captured from the reference decoder* (apart from the fact
> that said reference somehow lacks the top part of the frame
> (copied over from the background frame)).
> 2. Given that the stride of the buffer may be different
> from the width of the video (despite one pixel taking one byte),
> there is a second check lateron making the first check redundant
> (if one returns immediately; a simple break at the second check
> is not sufficient, because it only exits the inner loop).
> 3. The check is based around the assumption of the stride being
> positive (it has this in common with the other check which
> will be fixed in a future commit).
> 4. Even after fixing the off-by-one error, the check in
> question is still triggered by all the non-background frames
> in the FATE sample as well as by A1100100.RL2. In all these
> cases, they use len == 255 and val == 128. For videos with
> background frame this just means "copy from the background
> frame", which would be done anyway lateron.* Yet for videos
> without it copying it is necessary to avoid leaving
> uninitialized parts in the video.
>
> *: Available in https://samples.mplayerhq.hu/game-formats/voyeur-rl2/
> **: Due to this, the code that copies the rest from the
> back frame is no longer executed for any of the samples
> available on the sample server. Given that these are only
> the files from the demo version of this game, I don't know
> whether this code is executed for any file in existence or not.
>
> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt at outlook.com>
> ---
> libavcodec/rl2.c | 5 +-
> tests/ref/fate/rl2 | 216 ++++++++++++++++++++++-----------------------
> 2 files changed, 109 insertions(+), 112 deletions(-)
>
> diff --git a/libavcodec/rl2.c b/libavcodec/rl2.c
> index 5dedb96266..2464ad59ac 100644
> --- a/libavcodec/rl2.c
> +++ b/libavcodec/rl2.c
> @@ -91,9 +91,6 @@ static void rl2_rle_decode(Rl2Context *s, const uint8_t *in, int size,
> break;
> }
>
> - if (len >= out_end - out)
> - break;
> -
> if (s->back_frame)
> val |= 0x80;
> else
> @@ -106,7 +103,7 @@ static void rl2_rle_decode(Rl2Context *s, const uint8_t *in, int size,
> out += stride_adj;
> line_end += stride;
> if (len >= out_end - out)
> - break;
> + return;
> }
> }
> }
> diff --git a/tests/ref/fate/rl2 b/tests/ref/fate/rl2
> index 9189822503..a3c6a75d05 100644
> --- a/tests/ref/fate/rl2
> +++ b/tests/ref/fate/rl2
> @@ -3,111 +3,111 @@
> #codec_id 0: rawvideo
> #dimensions 0: 320x200
> #sar 0: 0/1
> -0, 0, 0, 1, 192000, 0x7112a667
> -0, 1, 1, 1, 192000, 0x6936abf3
> -0, 2, 2, 1, 192000, 0xb1f08981
> -0, 3, 3, 1, 192000, 0x4ce7fece
> -0, 4, 4, 1, 192000, 0xf04decde
> -0, 5, 5, 1, 192000, 0x47fef74b
> -0, 6, 6, 1, 192000, 0x99b42ac2
> -0, 7, 7, 1, 192000, 0x3c4c419d
> -0, 8, 8, 1, 192000, 0x66bf5588
> -0, 9, 9, 1, 192000, 0xe1de4725
> -0, 10, 10, 1, 192000, 0x348b2af9
> -0, 11, 11, 1, 192000, 0x1ce73e83
> -0, 12, 12, 1, 192000, 0xcdaa6e02
> -0, 13, 13, 1, 192000, 0x370dc2ce
> -0, 14, 14, 1, 192000, 0x1e1e40fe
> -0, 15, 15, 1, 192000, 0x491470a4
> -0, 16, 16, 1, 192000, 0x88c43e9a
> -0, 17, 17, 1, 192000, 0x036f3f44
> -0, 18, 18, 1, 192000, 0xc8be7e25
> -0, 19, 19, 1, 192000, 0xbedb397d
> -0, 20, 20, 1, 192000, 0x97c410f4
> -0, 21, 21, 1, 192000, 0xc2c8225d
> -0, 22, 22, 1, 192000, 0xe396bccb
> -0, 23, 23, 1, 192000, 0x7e89c24c
> -0, 24, 24, 1, 192000, 0xb044954c
> -0, 25, 25, 1, 192000, 0x335890dd
> -0, 26, 26, 1, 192000, 0x58a457c0
> -0, 27, 27, 1, 192000, 0xeb0f4798
> -0, 28, 28, 1, 192000, 0x0bfc39a1
> -0, 29, 29, 1, 192000, 0x06a6905a
> -0, 30, 30, 1, 192000, 0x5300c99b
> -0, 31, 31, 1, 192000, 0x38f3f845
> -0, 32, 32, 1, 192000, 0x6afa3543
> -0, 33, 33, 1, 192000, 0x5106a8e0
> -0, 34, 34, 1, 192000, 0xc76f0dab
> -0, 35, 35, 1, 192000, 0x8efa6939
> -0, 36, 36, 1, 192000, 0x64ea23d7
> -0, 37, 37, 1, 192000, 0x421a2817
> -0, 38, 38, 1, 192000, 0xc854fa18
> -0, 39, 39, 1, 192000, 0x4a10d59c
> -0, 40, 40, 1, 192000, 0x72637829
> -0, 41, 41, 1, 192000, 0xdbbe2796
> -0, 42, 42, 1, 192000, 0xff742e6b
> -0, 43, 43, 1, 192000, 0xf94b9346
> -0, 44, 44, 1, 192000, 0xc90ea53c
> -0, 45, 45, 1, 192000, 0x177483bb
> -0, 46, 46, 1, 192000, 0x3510369c
> -0, 47, 47, 1, 192000, 0x501034bf
> -0, 48, 48, 1, 192000, 0x315c744b
> -0, 49, 49, 1, 192000, 0xdb5048ae
> -0, 50, 50, 1, 192000, 0x09a86221
> -0, 51, 51, 1, 192000, 0xb9c9568a
> -0, 52, 52, 1, 192000, 0x5eee665b
> -0, 53, 53, 1, 192000, 0xdef85517
> -0, 54, 54, 1, 192000, 0x7896b5ad
> -0, 55, 55, 1, 192000, 0x19fbad39
> -0, 56, 56, 1, 192000, 0x63358748
> -0, 57, 57, 1, 192000, 0xca0196a3
> -0, 58, 58, 1, 192000, 0x0b321da3
> -0, 59, 59, 1, 192000, 0xa07af07e
> -0, 60, 60, 1, 192000, 0x21f9310c
> -0, 61, 61, 1, 192000, 0x62d59874
> -0, 62, 62, 1, 192000, 0x021227b4
> -0, 63, 63, 1, 192000, 0x01dac0c2
> -0, 64, 64, 1, 192000, 0xf3c33a74
> -0, 65, 65, 1, 192000, 0xeebe83b7
> -0, 66, 66, 1, 192000, 0x9ec77f97
> -0, 67, 67, 1, 192000, 0xc91c2e37
> -0, 68, 68, 1, 192000, 0x7b58751d
> -0, 69, 69, 1, 192000, 0xb178dfbb
> -0, 70, 70, 1, 192000, 0x2a63b5be
> -0, 71, 71, 1, 192000, 0x44a407ac
> -0, 72, 72, 1, 192000, 0x9a8e17dd
> -0, 73, 73, 1, 192000, 0x5546f4c8
> -0, 74, 74, 1, 192000, 0xeda94586
> -0, 75, 75, 1, 192000, 0x941dfa6b
> -0, 76, 76, 1, 192000, 0x9772301d
> -0, 77, 77, 1, 192000, 0x8be16b16
> -0, 78, 78, 1, 192000, 0x26dd4496
> -0, 79, 79, 1, 192000, 0x27823797
> -0, 80, 80, 1, 192000, 0xacc914f7
> -0, 81, 81, 1, 192000, 0xcbb72c9b
> -0, 82, 82, 1, 192000, 0x4bd3391c
> -0, 83, 83, 1, 192000, 0x4e6adbfe
> -0, 84, 84, 1, 192000, 0x7a791c75
> -0, 85, 85, 1, 192000, 0xc4f59c94
> -0, 86, 86, 1, 192000, 0xc4f59c94
> -0, 87, 87, 1, 192000, 0x984a4a0b
> -0, 88, 88, 1, 192000, 0x3353f31f
> -0, 89, 89, 1, 192000, 0xa9d4dc5a
> -0, 90, 90, 1, 192000, 0xb33425d0
> -0, 91, 91, 1, 192000, 0x546d768a
> -0, 92, 92, 1, 192000, 0xfefbe5c9
> -0, 93, 93, 1, 192000, 0xbd6be61d
> -0, 94, 94, 1, 192000, 0xf5792731
> -0, 95, 95, 1, 192000, 0xccde0582
> -0, 96, 96, 1, 192000, 0x857d58ee
> -0, 97, 97, 1, 192000, 0xe914ce48
> -0, 98, 98, 1, 192000, 0x1f736298
> -0, 99, 99, 1, 192000, 0xec0b4846
> -0, 100, 100, 1, 192000, 0xe1422624
> -0, 101, 101, 1, 192000, 0x56e2b0e0
> -0, 102, 102, 1, 192000, 0xc4190640
> -0, 103, 103, 1, 192000, 0x7c461977
> -0, 104, 104, 1, 192000, 0x34b1d5e8
> -0, 105, 105, 1, 192000, 0xbdc70f7a
> -0, 106, 106, 1, 192000, 0xb466cd8d
> -0, 107, 107, 1, 192000, 0x0e86a04c
> +0, 0, 0, 1, 192000, 0x7324a772
> +0, 1, 1, 1, 192000, 0x6b48acfe
> +0, 2, 2, 1, 192000, 0xb4028a8c
> +0, 3, 3, 1, 192000, 0x4ef9ffd9
> +0, 4, 4, 1, 192000, 0xf25fede9
> +0, 5, 5, 1, 192000, 0x4a10f856
> +0, 6, 6, 1, 192000, 0x9bc62bcd
> +0, 7, 7, 1, 192000, 0x3e5e42a8
> +0, 8, 8, 1, 192000, 0x68d15693
> +0, 9, 9, 1, 192000, 0xe3f04830
> +0, 10, 10, 1, 192000, 0x369d2c04
> +0, 11, 11, 1, 192000, 0x1ef93f8e
> +0, 12, 12, 1, 192000, 0xcfbc6f0d
> +0, 13, 13, 1, 192000, 0x391fc3d9
> +0, 14, 14, 1, 192000, 0x20304209
> +0, 15, 15, 1, 192000, 0x4b2671af
> +0, 16, 16, 1, 192000, 0x8ad63fa5
> +0, 17, 17, 1, 192000, 0x0581404f
> +0, 18, 18, 1, 192000, 0xcad07f30
> +0, 19, 19, 1, 192000, 0xc0ed3a88
> +0, 20, 20, 1, 192000, 0x99d611ff
> +0, 21, 21, 1, 192000, 0xc4da2368
> +0, 22, 22, 1, 192000, 0xe5a8bdd6
> +0, 23, 23, 1, 192000, 0x809bc357
> +0, 24, 24, 1, 192000, 0xb2569657
> +0, 25, 25, 1, 192000, 0x356a91e8
> +0, 26, 26, 1, 192000, 0x5ab658cb
> +0, 27, 27, 1, 192000, 0xed2148a3
> +0, 28, 28, 1, 192000, 0x0e0e3aac
> +0, 29, 29, 1, 192000, 0x08b89165
> +0, 30, 30, 1, 192000, 0x5512caa6
> +0, 31, 31, 1, 192000, 0x3b05f950
> +0, 32, 32, 1, 192000, 0x6d0c364e
> +0, 33, 33, 1, 192000, 0x5318a9eb
> +0, 34, 34, 1, 192000, 0xc9810eb6
> +0, 35, 35, 1, 192000, 0x910c6a44
> +0, 36, 36, 1, 192000, 0x66fc24e2
> +0, 37, 37, 1, 192000, 0x442c2922
> +0, 38, 38, 1, 192000, 0xca66fb23
> +0, 39, 39, 1, 192000, 0x4c22d6a7
> +0, 40, 40, 1, 192000, 0x74757934
> +0, 41, 41, 1, 192000, 0xddd028a1
> +0, 42, 42, 1, 192000, 0x01952f76
> +0, 43, 43, 1, 192000, 0xfb5d9451
> +0, 44, 44, 1, 192000, 0xcb20a647
> +0, 45, 45, 1, 192000, 0x198684c6
> +0, 46, 46, 1, 192000, 0x372237a7
> +0, 47, 47, 1, 192000, 0x522235ca
> +0, 48, 48, 1, 192000, 0x336e7556
> +0, 49, 49, 1, 192000, 0xdd6249b9
> +0, 50, 50, 1, 192000, 0x0bba632c
> +0, 51, 51, 1, 192000, 0xbbdb5795
> +0, 52, 52, 1, 192000, 0x61006766
> +0, 53, 53, 1, 192000, 0xe10a5622
> +0, 54, 54, 1, 192000, 0x7aa8b6b8
> +0, 55, 55, 1, 192000, 0x1c0dae44
> +0, 56, 56, 1, 192000, 0x65478853
> +0, 57, 57, 1, 192000, 0xcc1397ae
> +0, 58, 58, 1, 192000, 0x0d441eae
> +0, 59, 59, 1, 192000, 0xa28cf189
> +0, 60, 60, 1, 192000, 0x240b3217
> +0, 61, 61, 1, 192000, 0x64e7997f
> +0, 62, 62, 1, 192000, 0x042428bf
> +0, 63, 63, 1, 192000, 0x03ecc1cd
> +0, 64, 64, 1, 192000, 0xf5d53b7f
> +0, 65, 65, 1, 192000, 0xf0d084c2
> +0, 66, 66, 1, 192000, 0xa0d980a2
> +0, 67, 67, 1, 192000, 0xcb2e2f42
> +0, 68, 68, 1, 192000, 0x7d6a7628
> +0, 69, 69, 1, 192000, 0xb38ae0c6
> +0, 70, 70, 1, 192000, 0x2c75b6c9
> +0, 71, 71, 1, 192000, 0x46b608b7
> +0, 72, 72, 1, 192000, 0x9ca018e8
> +0, 73, 73, 1, 192000, 0x5758f5d3
> +0, 74, 74, 1, 192000, 0xefbb4691
> +0, 75, 75, 1, 192000, 0x962ffb76
> +0, 76, 76, 1, 192000, 0x99843128
> +0, 77, 77, 1, 192000, 0x8df36c21
> +0, 78, 78, 1, 192000, 0x28ef45a1
> +0, 79, 79, 1, 192000, 0x299438a2
> +0, 80, 80, 1, 192000, 0xaedb1602
> +0, 81, 81, 1, 192000, 0xcdc92da6
> +0, 82, 82, 1, 192000, 0x4de53a27
> +0, 83, 83, 1, 192000, 0x507cdd09
> +0, 84, 84, 1, 192000, 0x7c8b1d80
> +0, 85, 85, 1, 192000, 0xc7079d9f
> +0, 86, 86, 1, 192000, 0xc7079d9f
> +0, 87, 87, 1, 192000, 0x9a5c4b16
> +0, 88, 88, 1, 192000, 0x3565f42a
> +0, 89, 89, 1, 192000, 0xabe6dd65
> +0, 90, 90, 1, 192000, 0xb54626db
> +0, 91, 91, 1, 192000, 0x567f7795
> +0, 92, 92, 1, 192000, 0x011ce6d4
> +0, 93, 93, 1, 192000, 0xbf7de728
> +0, 94, 94, 1, 192000, 0xf78b283c
> +0, 95, 95, 1, 192000, 0xcef0068d
> +0, 96, 96, 1, 192000, 0x878f59f9
> +0, 97, 97, 1, 192000, 0xeb26cf53
> +0, 98, 98, 1, 192000, 0x218563a3
> +0, 99, 99, 1, 192000, 0xee1d4951
> +0, 100, 100, 1, 192000, 0xe354272f
> +0, 101, 101, 1, 192000, 0x58f4b1eb
> +0, 102, 102, 1, 192000, 0xc62b074b
> +0, 103, 103, 1, 192000, 0x7e581a82
> +0, 104, 104, 1, 192000, 0x36c3d6f3
> +0, 105, 105, 1, 192000, 0xbfd91085
> +0, 106, 106, 1, 192000, 0xb678ce98
> +0, 107, 107, 1, 192000, 0x1098a157
Will apply this patchset tonight unless there are objections.
- Andreas
More information about the ffmpeg-devel
mailing list