[FFmpeg-devel] [PATCH] avcodec/motion_est: Fix xy indexing on range violation in ff_get_best_fcode()

[Andreas Rheinhardt]

Michael Niedermayer:
> This codepath seems untested, no testcases change
> 
> Found-by: <mkver>
> Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
> ---
>  libavcodec/motion_est.c | 8 +++++---
>  1 file changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/libavcodec/motion_est.c b/libavcodec/motion_est.c
> index 9c548c1567..4cf1afe888 100644
> --- a/libavcodec/motion_est.c
> +++ b/libavcodec/motion_est.c
> @@ -1621,9 +1621,11 @@ int ff_get_best_fcode(MpegEncContext * s, int16_t (*mv_table)[2], int type)
>                                       fcode_tab[my + MAX_MV]);
>                      int j;
>  
> -                        if(mx >= range || mx < -range ||
> -                           my >= range || my < -range)
> -                            continue;
> +                    if (mx >= range || mx < -range ||
> +                        my >= range || my < -range) {
> +                        xy++;
> +                        continue;
> +                    }
>  
>                      for(j=0; j<fcode && j<8; j++){
>                          if(s->pict_type==AV_PICTURE_TYPE_B || s->current_picture.mc_mb_var[xy] < s->current_picture.mb_var[xy])

It would be simpler if you added the xy++ to the outer-for-loop and
removed the other xy++.

- Andreas