[FFmpeg-cvslog] avfilter/vf_hqdn3d: Fix left-shift of negative numbers

Andreas Rheinhardt git at videolan.org
Wed Nov 9 19:02:01 EET 2022


ffmpeg | branch: master | Andreas Rheinhardt <andreas.rheinhardt at outlook.com> | Sun Nov  6 17:17:39 2022 +0100| [a7ccfdc0d7c9d5726654fd5efff459ac872165f1] | committer: Andreas Rheinhardt

avfilter/vf_hqdn3d: Fix left-shift of negative numbers

Affected filter-hqdn3d and filter-hqdn3d-sample FATE-tests.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt at outlook.com>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=a7ccfdc0d7c9d5726654fd5efff459ac872165f1
---

 libavfilter/vf_hqdn3d.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavfilter/vf_hqdn3d.c b/libavfilter/vf_hqdn3d.c
index ce2b9a2974..c796ea9ab4 100644
--- a/libavfilter/vf_hqdn3d.c
+++ b/libavfilter/vf_hqdn3d.c
@@ -180,7 +180,7 @@ static void precalc_coefs(double dist25, int depth, int16_t *ct)
     gamma = log(0.25) / log(1.0 - FFMIN(dist25,252.0)/255.0 - 0.00001);
 
     for (i = -(256<<LUT_BITS); i < 256<<LUT_BITS; i++) {
-        double f = ((i<<(9-LUT_BITS)) + (1<<(8-LUT_BITS)) - 1) / 512.0; // midpoint of the bin
+        double f = (i * (1 << (9-LUT_BITS)) + (1<<(8-LUT_BITS)) - 1) / 512.0; // midpoint of the bin
         simil = FFMAX(0, 1.0 - fabs(f) / 255.0);
         C = pow(simil, gamma) * 256.0 * f;
         ct[(256<<LUT_BITS)+i] = lrint(C);



More information about the ffmpeg-cvslog mailing list