[FFmpeg-devel] Benchmarking

[Michael Niedermayer]

On Tue, Dec 16, 2008 at 02:23:36AM +0000, Robert Swain wrote:
> 2008/12/15 Michael Niedermayer <michaelni at gmx.at>:
> 
> [...]
> 
> > In conclusion, we can see that the minimum is the most reliable way and
> > the average is the least reliable way to detect the faster.
> > We can also see that 3 samples are enough for almost all cases
> 
> I assume that means for future benchmark results it would be
> acceptable to run 3 or 5 repetitions and report the fastest.

id say, if someone runs 3 or 5, he could as well also post all.
One simply feels more confident when seeing all. For example when all 3 of
group A are lower than all 3 of group B then this gives more confidence
in A being faster than if for example 1 of A is faster than all of B
while 2 of A are slower than all of B ...

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

No great genius has ever existed without some touch of madness. -- Aristotle
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20081216/60a4e798/attachment.pgp>