Hypothesis Testing: Null and Alternative Hypothesis and Test Types
A hypothesis test weighs sample evidence against a null hypothesis H0 (a default claim, such as μ = 100) in favor of an alternative hypothesis H1; the p-value is the probability, if H0 were true, of a result at least as extreme as the one observed, and H0 is rejected when the p-value is below the significance level α. In brief: with n = 49, x̄ = 103, σ = 14 and H0: μ = 100, the standard error is 2, the test statistic is z = (103 − 100)/2 = 1.5 and the two-sided p-value is about 0.134, so at α = 0.05 we fail to reject H0.
Tests turn the question “is this difference real or just chance?” into a procedure. This guide explains how to state hypotheses, choose a one- or two-tailed test, compute a test statistic and a p-value, and which test to use for means, proportions, paired data and two groups, with worked examples and a table.
How do you state the null and alternative hypotheses?
The null hypothesis H0 is the claim of “no effect” or “no difference”, always containing equality (μ = μ0, p = p0). The alternative H1 is what the evidence is supposed to support. It can be two-sided (μ ≠ μ0) or one-sided (μ > μ0 or μ < μ0). The direction is chosen before looking at the data, from the question being asked.
Example 1: stating hypotheses
A machine should fill bags with 100 g on average. To check for any deviation: H0: μ = 100, H1: μ ≠ 100 (two-sided). To check whether customers are being shortchanged: H1: μ < 100 (one-sided).
What are the steps of a test?
- State H0, H1 and the significance level α (commonly 0.05).
- Compute the test statistic, which measures how many standard errors the estimate is from the value under H0.
- Find the p-value (or compare with the critical value).
- Decide: reject H0 if p < α, otherwise fail to reject; then state the conclusion in the context of the problem.
Failing to reject does not prove H0; it only says that the data are not inconsistent with it.
How does a z-test for a mean work?
When σ is known (or n is large), the statistic is z = (x̄ − μ0)/(σ/√n), approximately standard normal under H0. The two-sided p-value is 2 · P(Z > |z|).
Example 2: the bag-filling machine
H0: μ = 100. n = 49, x̄ = 103, σ = 14. z = 3/2 = 1.5. P(Z > 1.5) = 0.0668, so the two-sided p-value is 0.1336 > 0.05: fail to reject. Check with the interval: 103 ± 1.96 · 2 = (99.08, 106.92) contains 100, which agrees. A test at level α rejects exactly when the (1 − α) confidence interval excludes μ0.
Example 3: a one-sided version
With H1: μ > 100, the p-value is 0.0668. This is greater than 0.05 (fail to reject), but smaller than 0.10 (reject at the 10% level). The conclusion depends on α, which must be fixed before the analysis.
How does a t-test work?
When σ is unknown, replace it by the sample standard deviation s: t = (x̄ − μ0)/(s/√n), with n − 1 degrees of freedom. The t distribution has heavier tails than the normal for small n.
Example 4: a small sample
H0: μ = 50, n = 16, x̄ = 52, s = 8. The standard error is 8/4 = 2 and t = 1.0 with 15 degrees of freedom. The two-sided critical value at α = 0.05 is 2.131, and 1.0 is far below it: fail to reject.
Example 5: paired data
Twenty-five people are measured before and after a training. The differences have the mean 2.5 and the standard deviation 5. Testing H0: mean difference = 0: t = 2.5/(5/√25) = 2.5 with 24 degrees of freedom. The critical value is 2.064, so reject H0: the training seems to make a difference.
How do you test a proportion?
For H0: p = p0 and a sample proportion p̂ from n trials: z = (p̂ − p0)/√(p0(1 − p0)/n). The standard error is computed with p0, the value under H0 (compare the interval for a proportion, which uses p̂).
Example 6: a possibly biased coin
100 flips give 62 heads. H0: p = 0.5. The standard error is √(0.25/100) = 0.05, so z = 0.12/0.05 = 2.4. The two-sided p-value is 2 · 0.0082 = 0.0164 < 0.05: reject H0. The data are unlikely for a fair coin, though not impossible.
Which test types exist?
- One-sample z-test and t-test for a mean.
- Two-sample t-test for the difference of the means of two independent groups, and the paired t-test for matched data.
- z-test for one or two proportions.
- Chi-square tests for goodness of fit and independence in tables of counts.
- F-test and ANOVA for comparing variances or several means.
Tests that assume a normal population or large samples are called parametric. When these assumptions fail, rank-based (nonparametric) tests such as the sign test or the Mann–Whitney test are used.
Table of test types
| Test | Use when | Statistic | Reference distribution |
|---|---|---|---|
| one-sample z | mean, σ known or n large | (x̄ − μ0)/(σ/√n) | standard normal |
| one-sample t | mean, σ unknown | (x̄ − μ0)/(s/√n) | t, n − 1 df |
| paired t | before/after data | d̄/(sd/√n) | t, n − 1 df |
| proportion z | one proportion | (p̂ − p0)/√(p0(1 − p0)/n) | standard normal |
| chi-square | counts in categories | Σ (O − E)2/E | chi-square |
Where does hypothesis testing come from?
Ronald Fisher popularized significance testing and the p-value in the 1920s, while Jerzy Neyman and Egon Pearson developed in the 1930s the framework of two hypotheses. Gosset’s t-test dates from 1908. Modern practice mixes the two traditions, which is a source of frequent misunderstandings.
Which mistakes are common?
Reading the p-value as the probability that H0 is true
The p-value is computed assuming H0. It is not the probability of H0.
Choosing the tail after seeing the data
The direction of H1 must follow from the research question, not from the sign of the observed difference.
Equating statistical and practical significance
With a huge sample, a trivial difference can be highly significant. Report the size of the effect and a confidence interval as well.
Concluding that H0 is proven
“Fail to reject” means insufficient evidence against H0, not evidence for it.