[FFmpeg-devel] [PATCH 2/9] fate: add 10bits YUV4:2:2 test

[James Almer]

On 10/13/2015 5:06 PM, Christophe Gisquet wrote:
> Hi,
> 
> 2015-10-13 21:41 GMT+02:00 James Almer <jamrial at gmail.com>:
>> > This test is failing on pretty much every fate client. Valgrind seems to
>> > complain about uninitialized values.
>> > http://fate.ffmpeg.org/report.cgi?time=20151013040721&slot=x86_64-archlinux-gcc-valgrindundef
> Can someone test the following patch? I don't think the fate results change.
> 
> -- Christophe
> 
> 
> 0001-dnxhdenc-fix-access-outside-of-image.patch
> 
> 
> From 14c1cd9c71af08d2875f6804dc286bc6fcfbf74e Mon Sep 17 00:00:00 2001
> From: Christophe Gisquet <christophe.gisquet at gmail.com>
> Date: Tue, 13 Oct 2015 22:02:41 +0200
> Subject: [PATCH] dnxhdenc: fix access outside of image
> 
> This is the same test as for the 8bit case.
> ---
>  libavcodec/dnxhdenc.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/libavcodec/dnxhdenc.c b/libavcodec/dnxhdenc.c
> index 1e14b8c..f4a7b98 100644
> --- a/libavcodec/dnxhdenc.c
> +++ b/libavcodec/dnxhdenc.c
> @@ -776,11 +776,13 @@ static int dnxhd_mb_var_thread(AVCodecContext *avctx, void *arg,
>              unsigned mb  = mb_y * ctx->m.mb_width + mb_x;
>              int sum = 0;
>              int sqsum = 0;
> +            int bw = FFMIN(avctx->width - 16 * mb_x, 16);
> +            int bh = FFMIN((avctx->height >> ctx->interlaced) - 16 * mb_y, 16);
>              int mean, sqmean;
>              int i, j;
>              // Macroblocks are 16x16 pixels, unlike DCT blocks which are 8x8.
> -            for (i = 0; i < 16; ++i) {
> -                for (j = 0; j < 16; ++j) {
> +            for (i = 0; i < bh; ++i) {
> +                for (j = 0; j < bw; ++j) {
>                      // Turn 16-bit pixels into 10-bit ones.
>                      int const sample = (unsigned) pix[j] >> 6;
>                      sum   += sample;
> -- 2.6.0

Fixes the failure and valgrind errors, so applied.