[MPlayer-dev-eng] [PATCH] fix aspect ratio calculations broken by new av_cmp_q

[Andrew Wason]

On Tue, Oct 19, 2010 at 2:35 AM, Reimar Döffinger
<Reimar.Doeffinger at gmx.de> wrote:
> In that case the patch is working exactly as intended.
> The container claims one aspect, the video stream claims another.
> Currently MPlayer is supposed to go with the value in the video stream,
> since otherwise resolution/aspect changes won't work right.


But isn't the opposite happening? In this video the AVStream aspect is
16:9, the AVCodecContext aspect is 4:3. MPlayer used to play it as
16:9. After av_cmp_q changed MPlayer plays it as 4:3.

sh->aspect is being computed as 1.777 from the AVStream aspect in
demux_lavf.c:handle_stream() - here MPlayer is preferring the AVStream
aspect over the AVCodecContext aspect.

if(st->sample_aspect_ratio.num)
    sh_video->aspect = codec->width  * st->sample_aspect_ratio.num
         / (float)(codec->height * st->sample_aspect_ratio.den);
else
    sh_video->aspect=codec->width  * codec->sample_aspect_ratio.num
         / (float)(codec->height * codec->sample_aspect_ratio.den);

For this video codec->sample_aspect_ratio is {8,9} and
st->sample_aspect_ratio is {1920,1620}, and codec width/height are
720,480.  So MPlayer is preferring the AVStream aspect and computing
sh->aspect as 1.777 instead of 1.333.

But then in vd_ffmpeg.c:init_vo() it overwrites sh->aspect with 1.333
(computed from the AVCodecContext aspect) due to the change in
av_cmp_q.

In any case, it seems like aspect changes would still work with my
patch because the first time init_vo is called
last_sample_aspect_ratio is not a valid aspect (either 0,0 or 0,1) and
sh->aspect has already been initialized properly in
demux_lavf.c:handle_stream( - so we only need to worry about
subsequent changes to avctx->sample_aspect_ratio which will be
properly detected because we will have set last_sample_aspect_ratio to
the initial avctx->sample_aspect_ratio we saw. The patch only prevents
resetting sh->aspect the first time (when last_sample_aspect_ratio has
not yet been initialized), it still allows last_sample_aspect_ratio to
be set to the last avctx->sample_aspect_ratio

Andrew