Hypothesis Testing with Two Samples

Summary :

Many questions compare two groups rather than one. This chapter extends hypothesis testing to two samples, showing how to test the difference between two population means when standard deviations are known or unknown, how to compare two population proportions, and how to handle matched or paired samples where observations come in pairs.

Comparing Two Population Means

When two groups are compared, the hypotheses concern the difference between their population means. If the population standard deviations are known, the normal distribution is used; if they are unknown and estimated from the samples, the Student t distribution is used instead. The test statistic measures how far the observed difference between the two sample means lies from the value stated in the null hypothesis, usually a difference of zero, relative to the variability of the data.

Comparing Two Proportions

To compare two population proportions, such as the success rates of two treatments, the difference between the two sample proportions is examined. Under the assumption that the proportions are equal, the sampling distribution of their difference is approximately normal for large samples, so a z-based test applies. As with means, the observed difference is standardised and turned into a p-value that measures how surprising the result would be if the two proportions were truly equal.

Matched or Paired Samples

Sometimes the two samples are not independent but naturally paired, such as the same subjects measured before and after a treatment. In this case the analysis works with the differences within each pair rather than the two groups separately. The set of paired differences is then treated as a single sample, and a one-sample test on those differences checks whether the average change differs from zero, controlling for variation between individuals.