[MPlayer-dev-eng] [PATCH] fix aspect ratio calculations broken by new av_cmp_q

[Andrew Wason]

On Mon, Oct 18, 2010 at 3:42 PM, Reimar Döffinger
<Reimar.Doeffinger at gmx.de> wrote:
>
> I can see two issues in the current code, but I think your patch rather
> adds a third actually.
> Please check if below patch seems ok.


Initializing ctx->last_sample_aspect_ratio.den = 1 was what I had
tried initially, but it does not correct some of my samples.

One of my samples has avctx->sample_aspect_ratio == {8, 9} and since
ctx->last_sample_aspect_ratio == {0, 1} then av_cmp_q returns 1
instead of 0 and the video is displayed with the wrong aspect ratio.

Hmm, I can't seem to upload a sample to mplayer incoming, but these
are the values I'm seeing in gdb for this sample with your patch in
libmpcodecs/vd_ffmpeg.c:init_vo

(gdb) p aspect
$8 = 1.33333337
(gdb) p sh->aspect
$9 = 1.77777779
(gdb) p avctx->sample_aspect_ratio
$10 = {
  num = 8,
  den = 9
}
(gdb) p ctx->last_sample_aspect_ratio
$11 = {
  num = 0,
  den = 1
}

The video should be displayed with 1.7777 but is using 1.3333

Attached patch keeps your check for aspect!=0 but keeps my
initialization of ctx->last_sample_aspect_ratio - this works with all
the samples I'm testing with.


Andrew
-------------- next part --------------
Index: libmpcodecs/vd_ffmpeg.c
===================================================================
--- libmpcodecs/vd_ffmpeg.c	(revision 32511)
+++ libmpcodecs/vd_ffmpeg.c	(working copy)
@@ -536,6 +536,9 @@
     float aspect= av_q2d(avctx->sample_aspect_ratio) * avctx->width / avctx->height;
     int width, height;
 
+    if (ctx->last_sample_aspect_ratio.den == 0)
+        ctx->last_sample_aspect_ratio = avctx->sample_aspect_ratio;
+
     width = avctx->width;
     height = avctx->height;
 
@@ -560,8 +563,9 @@
         set_format_params(avctx, avctx->pix_fmt);
         mp_msg(MSGT_DECVIDEO, MSGL_V, "[ffmpeg] aspect_ratio: %f\n", aspect);
         if (sh->aspect == 0 ||
+            (aspect != 0 &&
             av_cmp_q(avctx->sample_aspect_ratio,
-                     ctx->last_sample_aspect_ratio))
+                     ctx->last_sample_aspect_ratio)))
             sh->aspect = aspect;
         ctx->last_sample_aspect_ratio = avctx->sample_aspect_ratio;
         sh->disp_w = width;