[MPlayer-dev-eng] Re: Compile options

[Andrew Savchenko]

On 21 Sep 2006, 03:29 Trent Piepho wrote:
> On Wed, 20 Sep 2006, Andrew Savchenko wrote:
> > I've made some tests for 3 movie types: H264 HD
> > (http://images.apple.com/movies/wb/the_fountain/the_fountain-tsr_h4
> >80p.mov), x264 and mpeg4; and for 4 sets of compile options: general
> > -O4 option,
>
> Is there a sample to download for the mpeg4 test?

I can upload a file which I use for mpeg4 testing, it is about 68 Mb and 
3 minutes length. But only developers have read access on mplayerhq's 
ftp, so I don't know what to do.

> > There are my results, mplayer-dev-SVN-r19909-4.0.2 was used:
> > ==========================================================
> > Results for H264 HD:
> > ==> General -O4 build:
> >
> > vc: (4.724 \pm 0.009)s
> > vo: (25.117 \pm 0.013)s
> > user: 0m(19.842 \pm 0.054)s
>
> I'm not sure I follow the syntax here.  Does "\pm 0.009" mean that
> the sample standard deviation is .009 seconds, or the sample
> variance?  Or is it the variance/standard deviation of the sample
> mean?

It is variance/standart deviation of the sample mean.

> You can not properly compare the results with just the mean and
> variance. You need to also know the number of samples to know how
> many degrees of freedom there are in the t distribution.

Yes, of course. I used 10 samples: each file was tested 10 times, then I 
calculate error using t-distribution (also known as Student's 
coefficients) for reliability 0.95: t(0.95,10)=2.26.
Mean deviation was calculated as (LaTeX notation):

\[ \sigma = \sqrt {\frac { \sum_{i=1}^n \left( <x> - x_i \right)^2 } {n 
\cdot (n-1)}} = \sqrt { \frac {<x^2> - <x>^2 } {n-1}} \]

where <x> is mean value of x (CPU time in this case) and <x^2> is mean 
of square value x^2; the last equation follows from well-known 
porperties of dispersion for normal distribution. n is number of samles 
(10 in my case).

Errors were calculated in this way:
error = sigma * t(0.95,n); where n is number of samples.
Of course, value for t-distribution can be calculated with numerical 
integration, but I used well-known value from tables for simplicity of 
data processing.

Thus number of samples is taken into account. I can append to a mail a 
full data processing logs if you are interesting in it.

I have such small error only with help of statistics (10 measurements). 
I can't see any reason to increase number of samples, because system 
clock also have an error, and I don't know exactly its value (according 
to ntpdate it is about \pm 0.02 sec with respect to adjtimex 
adjustment). Of course, ntp (as well as xscreensaver and similar stuff) 
was disabled during testing.

> If you are trying to compare multiple groups at once, then you might
> want to try using ANOVA instead of multiple t tests.  In that case,
> you also need to know the number of samples in each group.

But why ANOVA is better than multiple t-tests? Imho t-tests are enough 
for reliable result.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 191 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/mplayer-dev-eng/attachments/20060921/1812a0a5/attachment.pgp>