[MPlayer-dev-eng] [PATCH] fix aspect ratio calculations broken by new av_cmp_q

[Andrew Wason]

On Mon, Oct 18, 2010 at 5:45 PM, Reimar Döffinger
<Reimar.Doeffinger at gmx.de> wrote:
> On Mon, Oct 18, 2010 at 05:25:41PM -0400, Andrew Wason wrote:
>> One of my samples has avctx->sample_aspect_ratio == {8, 9} and since
>> ctx->last_sample_aspect_ratio == {0, 1} then av_cmp_q returns 1
>> instead of 0 and the video is displayed with the wrong aspect ratio.
>
> Why should it return 0?!?


Perhaps it shouldn't, but returning 0 fixes all the samples broken by
the change to av_cmp_q.


> I'd say what you are seeing is that the bug that your patch adds
> (or I guess restores to be pedantic) works in your favour by disabling
> aspect updates in some cases (too early to be specific).
> Maybe below patch works.


No, this patch doesn't work.

sh_video->aspect is being correctly computed as 1.777 in
demux_lavf.c:handle_stream()

In vd_ffmpeg.c:init_vo, the local variable 'aspect' is 1.333.
av_cmp_q used to return 0 when ctx->last_sample_aspect_ratio was {0,0}
and so sh->aspect was left as 1.777

Now av_cmp_q returns INT_MIN when ctx->last_sample_aspect_ratio is
{0,0}, and with your patch it returns 1 when it is {0,1} - and so in
both cases we overwrite 1.777 with 1.333.

So to get the old behavior, av_cmp_q should return 0 the first time
init_vo is called and so the way I was doing that was to initialize
ctx->last_sample_aspect_ratio to avctx->sample_aspect_ratio.

Attached is another patch that leaves last_sample_aspect_ratio
initialized to {0,0} and avoids using av_cmp_q if
last_sample_aspect_ratio has not yet been reinitialized.  This works
with all my samples.


I was able to trim one of the problematic samples down to 1M and put
it here for download:

https://docs.google.com/leaf?id=0Bw6Va_daKFKMYmVkMGMxYWMtMjVjMi00NGMxLWFiMjktYjEyOGNiYWNlMzAw&hl=en&authkey=CKvBt4gK

Andrew
-------------- next part --------------
Index: libmpcodecs/vd_ffmpeg.c
===================================================================
--- libmpcodecs/vd_ffmpeg.c	(revision 32511)
+++ libmpcodecs/vd_ffmpeg.c	(working copy)
@@ -560,8 +560,9 @@
         set_format_params(avctx, avctx->pix_fmt);
         mp_msg(MSGT_DECVIDEO, MSGL_V, "[ffmpeg] aspect_ratio: %f\n", aspect);
         if (sh->aspect == 0 ||
-            av_cmp_q(avctx->sample_aspect_ratio,
-                     ctx->last_sample_aspect_ratio))
+            (ctx->last_sample_aspect_ratio.den != 0 &&
+             av_cmp_q(avctx->sample_aspect_ratio,
+                      ctx->last_sample_aspect_ratio)))
             sh->aspect = aspect;
         ctx->last_sample_aspect_ratio = avctx->sample_aspect_ratio;
         sh->disp_w = width;