[FFmpeg-devel] [PATCH] Fix VP3 dequantization matrices when an IDCT permutation is used

[Aurelien Jacobs]

David Conrad wrote:

> Hi,
> 
> It appears that the reason why the VP3 SSE/MMX IDCTs weren't bitexact  
> to the C version was because the dct coefficients weren't being  
> dequantized correctly when a permutation was needed. The coefficients  
> are permutated correctly in unpack_vlcs() but the dequantization  
> matrices are built assuming no permutation. This fix allows both the  
> SSE2 and MMX IDCTs produce bitexact output for the VP3 and Theora  
> samples I have.
> 
> [...]
> 
> --- a/libavcodec/i386/dsputil_mmx.c
> +++ b/libavcodec/i386/dsputil_mmx.c
> @@ -2528,9 +2528,7 @@ void dsputil_init_mmx(DSPContext* c, AVCodecContext *avctx)
>                  c->idct_permutation_type= FF_LIBMPEG2_IDCT_PERM;
>  #endif
>              }else if((ENABLE_VP3_DECODER || ENABLE_VP5_DECODER || ENABLE_VP6_DECODER) &&
> -                     idct_algo==FF_IDCT_VP3 &&
> -                     avctx->codec->id!=CODEC_ID_THEORA &&
> -                     !(avctx->flags & CODEC_FLAG_BITEXACT)){
> +                     idct_algo==FF_IDCT_VP3){

As you enable this code for theora, you also must add a
'|| ENABLE_THEORA_DECODER' to the above list, to ensure this code
won't be disabled when only the theora decoder is compiled in.

Aurel