[MPlayer-users] lavc psnr mixup

[Loren Merritt]

On Tue, 7 Sep 2004, D Richard Felker III wrote:
> On Tue, Sep 07, 2004 at 03:48:01PM -0700, Loren Merritt wrote:
>
>> The PSNR values in the psnr_*.log generated by mencoder/lavc seem to be
>> matched up to the wrong frame numbers. This occurs consistently on all
>> input videos and all lavcopts, including different vcodec's.
>> The offset is (3 + vmax_b_frames), and happens in display order, even
>> though stats are output in encoded order.
>> The other stats (qp, size, frame type) are matched to the correct
>> frame number.
>> The PSNRs that get bumped off the end due to this offset are not used
>> when calculating the PSNR summary printed by mencoder, but the "inf"s
>> inserted at the beginning are also not counted in that summary.
>
> rotfl!! i bet this bug causes the infinities (which get rounded down
> to 100 or something) to get averaged into the final psnr, giving the
> illusion that B frames improve quality. i also bet that if you fix
> this bug, it will be revealed that B frames suck and greatly reduce
> psnr... :)

That's not it. 2 reasons:

1)
You could take an arithmetic mean of the per-frame psnrs, and the result
would be completely thrown off by a few frames with infinite psnr. (And it
really is infinite, not just big. If you have to truncate it to be <100 or
any other arbitrary limit, that just highlites to bogusness of the math.)

But unlike some other encoders (*coughx264cough*), ffmpeg and mencoder
do the right thing with global psnr: First take the mean squared error,
then calculate psnr from that. This way, a few frames of 0 error
(inf psnr) just slightly reduce the mean error.

2)
The extra infinities are not even included in the final psnr:

(Note that 48.31 is the correct psnr average of 4x 47.07 and 4x 50.06, but
it should include the extra 3x 50.06 that got bumped off the end.)

$> mencoder in.yuv -ovc lavc -lavcopts vqscale=2:psnr:vmax_b_frames=0
[...]
Video stream: 1652.095 kbit/s  (206511 bps)  size: 94746 bytes  0.459 secs  11 frames
PSNR: Y:48.07, Cb:48.58, Cr:49.13, All:48.31
$> cat psnr_212659.log
     0, 2.00,  73787, inf, inf, inf, inf I
     1, 2.00,    236, inf, inf, inf, inf P
     2, 2.00,    187, inf, inf, inf, inf P
     3, 2.00,    188, 46.63, 47.40, 48.98, 47.07 P
     4, 2.00,  18991, 46.64, 47.40, 48.98, 47.07 I
     5, 2.00,    460, 46.64, 47.40, 48.98, 47.08 P
     6, 2.00,    193, 46.64, 47.40, 48.98, 47.08 P
     7, 2.00,    178, 50.01, 50.27, 49.89, 50.03 P
     8, 2.00,    176, 50.05, 50.28, 49.91, 50.06 P
     9, 2.00,    175, 50.05, 50.28, 49.91, 50.06 P
    10, 2.00,    175, 50.05, 50.28, 49.91, 50.06 P

$> mencoder in.yuv -ovc lavc -lavcopts vqscale=2:psnr:vmax_b_frames=2
[...]
Video stream: 1686.882 kbit/s  (210860 bps)  size: 96741 bytes  0.459 secs  11 frames
PSNR: Y:48.07, Cb:48.58, Cr:49.13, All:48.32
$> cat psnr_212744.log
     0, 0.00,      0, inf, inf, inf, inf ?
     0, 0.00,      0, inf, inf, inf, inf ?
     0, 2.00,  73788, inf, inf, inf, inf I
     1, 2.00,    236, inf, inf, inf, inf P
     2, 2.00,     29, inf, inf, inf, inf B
     3, 2.00,     26, inf, inf, inf, inf B
     4, 2.00,  18992, 46.64, 47.40, 48.98, 47.08 I
     5, 2.00,   1368, inf, inf, inf, inf B
     6, 2.00,   1368, 46.64, 47.40, 48.98, 47.08 B
     7, 2.00,    460, 50.07, 50.27, 49.91, 50.08 P
     8, 2.00,    148, 46.63, 47.40, 48.98, 47.07 B
     9, 2.00,    133, 50.07, 50.27, 49.91, 50.08 B
    10, 2.00,    193, 50.07, 50.28, 49.91, 50.08 P


... That said, while I do find that B-frames improve both psnr and
visual quality in most real movies, I have come to agree that they still
suck and should be replaced by some other 2nd order temporal prediction
(3D wavelets?).

--Loren Merritt