[FFmpeg-devel] [PATCH] lavc/vvc: Fix check whether QG is in first tile col

[Frank Plowman]

The second part of this condition is intended to check whether the
current quantisation group is in the first CTU column of the current
tile.  The issue is that ctb_to_col_bd gives the x-ordinate of the first
column of the current tile *in CTUs*, while xQg gives the x-ordinate of
the quantisation group *in samples*.  Rectify this by shifting xQg by
ctb_log2_size to get xQg in CTUs before comparing.

Fixes FFVVC issues #201 and #203.
---
 libavcodec/vvc/vvc_ctu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/vvc/vvc_ctu.c b/libavcodec/vvc/vvc_ctu.c
index 519bd1ba76..8ba12c8d9f 100644
--- a/libavcodec/vvc/vvc_ctu.c
+++ b/libavcodec/vvc/vvc_ctu.c
@@ -96,7 +96,7 @@ static int get_qp_y_pred(const VVCLocalContext *lc)
     if (lc->na.cand_up) {
         const int first_qg_in_ctu = !(xQg & ctb_size_mask) &&  !(yQg & ctb_size_mask);
         const int qPy_up          = fc->tab.qp[LUMA][x_cb + (y_cb - 1) * min_cb_width];
-        if (first_qg_in_ctu && pps->ctb_to_col_bd[xQg >> ctb_log2_size] == xQg)
+        if (first_qg_in_ctu && pps->ctb_to_col_bd[xQg >> ctb_log2_size] == xQg >> ctb_log2_size)
             return qPy_up;
     }
 
-- 
2.42.0