Question

How can you determine if a difference between the means of two samples is significant?

Answer 1

You form a new statistic which is the difference between the two means which allows you to ask significance questions about it.

Explanation:

In this question we want to know about the difference of two means. This is a function of random variables, i.e.

`mu_(1-2)=mu_1-mu_2`

or, in our case the function is of the sample means:

`m_(1-2)=m_1-m_2`

There are many assumptions that go into the next steps (see this link for details Stat Trek: difference between means ) but for now lets assume that the two distributions we are sampling are approximately normal, and that we only have relatively few sampling points for each (otherwise we would be certain of the values of the means and therefore the difference).

Given this, we can calculate the sample variance of the difference of means from the sample variances of the two samples:

`s_(1-2)^2=s_1^2/n_1+s_2^2/n_2`

Note that what goes into this calculation is the sample variances divided by the number of points, which is the variance of the calculated means and follows the expected form of the central limit theorem (Variance of sample mean ).

Before we ask questions about the new distribution using t-statistics, we need to know the degrees of freedom which can be approximated from (Welch–Satterthwaite equation ):

`D.F.=(s_1^2/n_1+s_2^2/n_2)^2/((s_1^2/n_1)^2/(n_1-1)+(s_2^2/n_2)^2/(n_2-1))`

This equation allows for a different significance of each point from the two distributions based on their variance and a different number of samples from each. If the distributions have the same variance and we take the same number of samples, `n`, from each, this simplifies to `D.F.=2n-2`

Given all of these, we can use the students-t distribution to ask questions about the probability of the statistic `m_(1-2)` taking on specific values using:

`t = [ m_(1-2) - d ] / s_(1-2)`

Where d is the proposed distance between the two means.