[MPlayer-dev-eng] Re: Compile options

[Andrew Savchenko]

Greetings!

> > Howere, how about mean deviation value?
> > Can you provide results with errors?
> > Result without error is meaningless.
>
> One almost never sees a computer benchmark done by someone who knows
> anything about statistics.

Now you can see some.

> Even well-known review sites think they 
> can run a benchmark just once and then compare it, without any idea
> of the benchmark's variance.

And it is very unwell. Results can't be reliable without errors: lets 
assume you have measured two distances A=1.5m, B=1.6m but with error 
0.2m (for reliability 0.95). In this case you can't say that B is 
larger A. If error is unknown you have no guaranty that it is not too 
large.

Similar things are in computer benchmarking, because there are a couple 
of unpredictable factors: disk cache, system processes and so on. Every 
physicist will say you that every measurement without error values are 
meaningless.

Statisticts isn't difficult to use it in benchmarking.

> To get better benchmarks, this script might help.  Do something like:
>
> for i in `seq 1 5`;do time mplayer -quiet -benchmark ... 2>&1;done|
> mpbench.pl foo
>
> You'll get output like this:
> vc.foo <- c( 7.643,8.068,7.849,7.863,7.818 );
> vo.foo <- c( 0.005,0.005,0.005,0.005,0.005 );
> sys.foo <- c( 0.393,0.421,0.411,0.408,0.406 );
> user.foo <- c( 7.97,8.37,8.16,8.17,8.17 );
> elapsed.foo <- c( 8.11,8.58,8.34,8.36,8.31 );
>
> e.g, all the different kinds of times for each run in a list.  If
> you've used the open source statistics package R, you might notice
> that this output could be cut and pasted into R.  Then one might
> analyse the data with tools like hist(), boxplot(), t.test(),
> wilcox.test(), and so on.

Thanks a lot, but in this case calculations are very simple and I've 
already made them by my own.

> Some useful links google turned up:
> http://www.stat.psu.edu/~dhunter/R/2006test.html
> http://www.math.csi.cuny.edu/Statistics/R/simpleR/stat011.html

Thanks. Just FYI I preffer to use following scientific project which is 
aimed for sophisticated numerical data analysing (it is very usable in 
physics):
http://root.cern.ch
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/mplayer-dev-eng/attachments/20060917/5aa32d22/attachment.pgp>