[FFmpeg-devel] [PATCH] vp9: fix type of iadst4_1d intermediates.

[Clément Bœsch]

On Fri, Sep 04, 2015 at 05:02:01PM -0400, Ronald S. Bultje wrote:
> Fixes integer overflows for extreme coefficient values in 10/12bpp content.
> ---
>  libavcodec/vp9dsp_template.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libavcodec/vp9dsp_template.c b/libavcodec/vp9dsp_template.c
> index 9395a0c..4d810fe 100644
> --- a/libavcodec/vp9dsp_template.c
> +++ b/libavcodec/vp9dsp_template.c
> @@ -1186,7 +1186,7 @@ static av_always_inline void idct4_1d(const dctcoef *in, ptrdiff_t stride,
>  static av_always_inline void iadst4_1d(const dctcoef *in, ptrdiff_t stride,
>                                         dctcoef *out, int pass)
>  {
> -    int t0, t1, t2, t3;
> +    dctint t0, t1, t2, t3;
>  
>      t0 =  5283 * IN(0) + 15212 * IN(2) +  9929 * IN(3);
>      t1 =  9929 * IN(0) -  5283 * IN(2) - 15212 * IN(3);

not needed for iwht4_1d?

-- 
Clément B.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 473 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20150905/3096b379/attachment.sig>