[MPlayer-dev-eng] [PATCH] fix aspect ratio calculations broken by new av_cmp_q

[Michael Niedermayer]

On Mon, Oct 18, 2010 at 07:38:20PM -0400, Andrew Wason wrote:
> On Mon, Oct 18, 2010 at 5:45 PM, Reimar Döffinger
> <Reimar.Doeffinger at gmx.de> wrote:
> > On Mon, Oct 18, 2010 at 05:25:41PM -0400, Andrew Wason wrote:
> >> One of my samples has avctx->sample_aspect_ratio == {8, 9} and since
> >> ctx->last_sample_aspect_ratio == {0, 1} then av_cmp_q returns 1
> >> instead of 0 and the video is displayed with the wrong aspect ratio.
> >
> > Why should it return 0?!?
> 
> 
> Perhaps it shouldn't, but returning 0 fixes all the samples broken by
> the change to av_cmp_q.
> 
> 
> > I'd say what you are seeing is that the bug that your patch adds
> > (or I guess restores to be pedantic) works in your favour by disabling
> > aspect updates in some cases (too early to be specific).
> > Maybe below patch works.
> 
> 
> No, this patch doesn't work.
> 
> sh_video->aspect is being correctly computed as 1.777 in
> demux_lavf.c:handle_stream()
> 
> In vd_ffmpeg.c:init_vo, the local variable 'aspect' is 1.333.
> av_cmp_q used to return 0 when ctx->last_sample_aspect_ratio was {0,0}
> and so sh->aspect was left as 1.777
> 
> Now av_cmp_q returns INT_MIN when ctx->last_sample_aspect_ratio is
> {0,0},

if someone wants to get back the old behavior 2*av_cmp_q() should do that
approximately
(or if one is nervous about signed overflows with 2U but then <0 / >0 need a
 cast back)

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Into a blind darkness they enter who follow after the Ignorance,
they as if into a greater darkness enter who devote themselves
to the Knowledge alone. -- Isha Upanishad
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/mplayer-dev-eng/attachments/20101021/8d3ce0e1/attachment.pgp>