Statistical Analysis and Evaluating Research Reliability

Statistical analysis in geography uses numbers to summarize data (descriptive statistics such as the mean and standard deviation) and to test whether patterns are likely to be real (inferential statistics such as Spearman’s rank correlation), and evaluating research reliability means judging how far the results can be trusted. In brief: the mean, median and standard deviation describe the center and spread of a data set. Spearman’s rank correlation coefficient (r_s) ranges from −1 (perfect negative) through 0 (none) to +1 (perfect positive), and is compared with a critical value for the sample size to find whether the relationship is statistically significant. Reliable research has an adequate sample, accurate methods and honest evaluation of errors and limits.

Statistics help to tell the difference between a real pattern and a chance result, and evaluation shows how much confidence to place in the conclusions. This guide explains the main descriptive and inferential methods, how to interpret significance, how to evaluate an investigation, compares the statistical tests in a table, works through standard deviation and Spearman calculations, outlines the history and corrects common mistakes.

Which descriptive statistics are used?

• Measures of central tendency: the mean (the sum divided by the number of values), the median (the middle value when the data are in order) and the mode (the most common value). The median is better when the data are skewed or have outliers, such as incomes or house prices.
• Measures of spread: the range (highest minus lowest), the interquartile range (the spread of the middle half of the data, the upper quartile minus the lower) and the standard deviation, which measures how far values typically lie from the mean. A small standard deviation means that values are clustered around the mean, and a large one shows wide scatter.
• Percentages, ratios and rates to compare groups of different size.

For a sample, the standard deviation is s = √[Σ(x − mean)² ÷ (n − 1)]. It is used to compare the variability of different data sets, such as pebble size at two sites, and with the mean it describes the distribution: in a normal distribution, about 68% of values lie within one standard deviation of the mean and 95% within two.

How do inferential tests work?

Inferential statistics ask whether a pattern found in the sample is likely to hold in the population, or could have occurred by chance. The process starts with a null hypothesis (H₀), which states that there is no relationship or difference. The test calculates a statistic, which is compared with a critical value from tables (or a p-value from software). If the result is more extreme than the critical value at a chosen significance level, usually 5% (p < 0.05), the null hypothesis is rejected, and the result is called statistically significant: there is less than a 5% probability that such a result would occur by chance if there was no relationship. A 1% level gives a stricter test. Significance is not the same as importance: a very large sample can find a significant but tiny effect.

Common tests in geography include:

• Spearman’s rank correlation: tests the strength and direction of a relationship between two variables measured on at least ordinal scales (for example, distance from the CBD and rent). It works with ranks, so it does not require a normal distribution and is less affected by outliers. The formula is r_s = 1 − [6 × Σd² ÷ (n × (n² − 1))], where d is the difference in rank for each pair, and n is the number of pairs.
• Pearson’s correlation coefficient: for linear relationships between two normally distributed numerical variables.
• Chi-squared test: compares observed and expected frequencies in categories, such as whether land use differs from what would be expected by chance.
• t-test and Mann-Whitney U test: compare the means or ranks of two groups, such as pebble size above and below a weir.
• Nearest neighbor analysis: tests whether points are clustered, random or regular.

How is reliability evaluated?

To judge a piece of research, consider several questions.

• Sample: was it large enough, chosen without bias and representative of the population? Were enough sites and times covered?
• Accuracy and precision of measurements: was equipment accurate and used consistently, with the same method at each site? Was there human error, such as reading a scale wrongly?
• Reliability and validity: reliability means that repeating the investigation would give similar results, and validity means that the method really measures what it claims to measure (for example, whether a float measures the mean velocity).
• Uncontrolled variables: were other factors, such as weather, time of day or recent rain, affecting the results?
• Anomalies: were outliers examined and treated openly?
• Bias and ethics: were questions neutral, were participants protected and was the researcher open about limitations?
• Secondary data: who collected them, when and for what purpose, and are they up to date and complete?

A good evaluation does not just list problems, but explains their effect on the conclusion and suggests improvements: a larger sample, more sites, repeating measurements at different times, using better equipment, or adding different methods to cross-check results (triangulation). It also states how confident the conclusion is, and avoids claiming more than the evidence shows.

Statistical tests compared

Worked example: standard deviation and Spearman’s rank

1. Mean: six pebble lengths (cm) at a site are 4, 6, 7, 9, 9 and 11. The sum is 46, so the mean is 46 ÷ 6 = 7.67 cm. The median is (7 + 9) ÷ 2 = 8 cm and the mode is 9 cm.
2. Standard deviation: the deviations from the mean are −3.67, −1.67, −0.67, 1.33, 1.33 and 3.33. Their squares are 13.44, 2.78, 0.44, 1.78, 1.78 and 11.11, with a sum of 31.33. For a sample, s = √(31.33 ÷ 5) = √6.27 = 2.50 cm.

For the Spearman test, take pebble lengths at eight sites, measured at distances of 2, 5, 8, 11, 14, 17, 20 and 23 km from the source. The hypothesis is that pebble size decreases downstream, and the null hypothesis is that there is no relationship.

3. Sum of d²: 49 + 25 + 9 + 1 + 4 + 4 + 25 + 49 = 166.
4. Coefficient: r_s = 1 − [6 × 166 ÷ (8 × (64 − 1))] = 1 − [996 ÷ 504] = 1 − 1.976 = −0.976.
5. Interpretation: the value is very close to −1, a very strong negative correlation: pebble size decreases as distance downstream increases.
6. Significance: for n = 8, the critical value at the 5% level is 0.738 and at the 1% level 0.881 (two-tailed). Since 0.976 (ignoring the sign) is greater than 0.881, the null hypothesis can be rejected at the 1% level: there is less than a 1% probability that such a result would occur by chance. This supports the hypothesis, but does not prove it for all rivers, and the small sample of 8 sites is a limitation.

How did the methods develop?

Karl Pearson developed the correlation coefficient in the 1890s, and Charles Spearman published his rank correlation in 1904. Ronald Fisher’s work in the 1920s and 1930s established significance testing, including the use of the 5% level. The “quantitative revolution” in geography in the 1950s and 1960s introduced statistical methods, and the use of computers allowed more complex analysis. Since then, critics have questioned the overuse of significance tests and the focus on p-values, and stressed effect sizes, confidence intervals and replication. The concern about a “replication crisis” in science has led to more attention to open data and transparent methods. Today, free software makes it simple to run tests, which increases the importance of understanding their assumptions.

Which mistakes are common?

Mistake 1: Assuming that a significant result proves a cause

Significance shows that a relationship is unlikely to be due to chance. It does not show why it exists, and other factors may be responsible.

Mistake 2: Ignoring the sign of the coefficient

The sign gives the direction. A coefficient of −0.98 is a very strong negative relationship, not a weak one.

Mistake 3: Using a test on too few observations

Small samples give weak and unstable results. Plan for enough data, and note this in the evaluation.

Mistake 4: Evaluating only by listing problems

A good evaluation explains how each limitation could affect the results and what changes would improve the study.