[FFmpeg-devel] [PATCH v2 1/3] avfilter/vf_bwdif: consider chroma subsampling when enforcing minimum dimensions

[Cosmin Stejerean]

On Nov 30, 2023, at 04:37, Thomas Mundt <tmundt75 at gmail.com> wrote:


Am Do., 30. Nov. 2023 um 01:23 Uhr schrieb Cosmin Stejerean via ffmpeg-devel <ffmpeg-devel at ffmpeg.org <mailto:ffmpeg-devel at ffmpeg.org> >:
From: Cosmin Stejerean <cosmin at cosmin.at <mailto:cosmin at cosmin.at> >

Fixes #10688

Signed-off-by: Cosmin Stejerean <cosmin at cosmin.at <mailto:cosmin at cosmin.at> >
---
 libavfilter/vf_bwdif.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/libavfilter/vf_bwdif.c b/libavfilter/vf_bwdif.c
index 137cd5ef13..80aa85a48b 100644
--- a/libavfilter/vf_bwdif.c
+++ b/libavfilter/vf_bwdif.c
@@ -191,12 +191,19 @@ static int config_props(AVFilterLink *link)
         return ret;
     }

-    if (link->w < 3 || link->h < 4) {
-        av_log(ctx, AV_LOG_ERROR, "Video of less than 3 columns or 4 lines is not supported\n");
+    const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(link->format);
+
+    int h = link->h;
+    int w = link->w;
+    int h_chroma = AV_CEIL_RSHIFT(h, desc->log2_chroma_h);
+    int w_chroma = AV_CEIL_RSHIFT(w, desc->log2_chroma_w);
+
+    if (w < 3 || w_chroma < 3 || h < 4 || h_chroma < 4) {
+        av_log(ctx, AV_LOG_ERROR, "Video with planes less than 3 columns or 4 lines is not supported\n");
         return AVERROR(EINVAL);
     }

-    yadif->csp = av_pix_fmt_desc_get(link->format);
+    yadif->csp = desc;
     yadif->filter = filter;
     ff_bwdif_init_filter_line(&s->dsp, yadif->csp->comp[0].depth);

I think mixed declarations are not allowed.
Also log2_chroma_w/h should never be negative, so why not just do:

if (AV_CEIL_RSHIFT(link->w,  yadif->csp->log2_chroma_w) < 3 ||
    AV_CEIL_RSHIFT(link->h,  yadif->csp->log2_chroma_h) < 4)


Thank you for the prompt feedback, makes a lot of sense to me, will update in v3. 

- Cosmin