[FFmpeg-cvslog] lavc/vvc: Prevent overflow in chroma QP derivation

[Frank Plowman]

ffmpeg | branch: master | Frank Plowman <post at frankplowman.com> | Sun Jun  9 12:17:26 2024 +0100| [d72a5fe719c01da07af30e4402a7c3cd994b4cfc] | committer: Nuo Mi

lavc/vvc: Prevent overflow in chroma QP derivation

On the top of p. 112 in VVC (09/2023):

It is a requirement of bitstream conformance that the values of
qpInVal[ i ][ j ] and qpOutVal[ i ][ j ] shall be in the range
of −QpBdOffset to 63, inclusive for i in the range of 0 to
numQpTables − 1, inclusive, and j in the range of 0 to
sps_num_points_in_qp_table_minus1[ i ] + 1, inclusive.

Additionally, don't discard the return code from sps_chroma_qp_table.

Signed-off-by: Frank Plowman <post at frankplowman.com>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=d72a5fe719c01da07af30e4402a7c3cd994b4cfc
---

 libavcodec/vvc/ps.c | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/libavcodec/vvc/ps.c b/libavcodec/vvc/ps.c
index 1b23675c98..92368eafc2 100644
--- a/libavcodec/vvc/ps.c
+++ b/libavcodec/vvc/ps.c
@@ -101,9 +101,14 @@ static int sps_chroma_qp_table(VVCSPS *sps)
 
         qp_out[0] = qp_in[0] = r->sps_qp_table_start_minus26[i] + 26;
         for (int j = 0; j < num_points_in_qp_table; j++ ) {
+            const uint8_t delta_qp_out = (r->sps_delta_qp_in_val_minus1[i][j] ^ r->sps_delta_qp_diff_val[i][j]);
             delta_qp_in[j] = r->sps_delta_qp_in_val_minus1[i][j] + 1;
+            // Note: we cannot check qp_{in,out}[j+1] here as qp_*[j] + delta_qp_*
+            //       may not fit in an 8-bit signed integer.
+            if (qp_in[j] + delta_qp_in[j] > 63 || qp_out[j] + delta_qp_out > 63)
+                return AVERROR(EINVAL);
             qp_in[j+1] = qp_in[j] + delta_qp_in[j];
-            qp_out[j+1] = qp_out[j] + (r->sps_delta_qp_in_val_minus1[i][j] ^ r->sps_delta_qp_diff_val[i][j]);
+            qp_out[j+1] = qp_out[j] + delta_qp_out;
         }
         sps->chroma_qp_table[i][qp_in[0] + off] = qp_out[0];
         for (int k = qp_in[0] - 1 + off; k >= 0; k--)
@@ -186,8 +191,11 @@ static int sps_derive(VVCSPS *sps, void *log_ctx)
     sps_inter(sps);
     sps_partition_constraints(sps);
     sps_ladf(sps);
-    if (r->sps_chroma_format_idc != 0)
-        sps_chroma_qp_table(sps);
+    if (r->sps_chroma_format_idc != 0) {
+        ret = sps_chroma_qp_table(sps);
+        if (ret < 0)
+            return ret;
+    }
 
     return 0;
 }