Skip to content

13-01: Correlation

Everything before this note examined one variable at a time (or one variable across groups). Correlation measures the strength and direction of the linear relationship between two quantitative variables.

Start With a Scatterplot — Always

Positive linear        Negative linear        No relationship        Curved

      ·  ·                ·                     ·   ·  ·             ·     ·
    ·  ·                 ·  ·                  ·  ·   ·               · · ·
  ·  ·                     ·  ·                 ·  ·  ·              ·     ·
 ·                            ·  ·             ·   ·                       
 r ≈ +0.9              r ≈ −0.9                r ≈ 0                r ≈ 0 (!)

The last panel is the reason the scatterplot comes first: r measures linear association only. A perfect parabola has r ≈ 0 and an obvious relationship.

Reading a scatterplot — four things:

  1. Direction — up-and-to-the-right (positive) or down (negative).
  2. Form — linear, curved, or none.
  3. Strength — how tightly the points cluster around a line.
  4. Outliers — points far from the pattern; these move r a great deal.

By convention the explanatory variable goes on the x axis and the response variable on y.

Pearson's Correlation Coefficient

                Σ (x − x̄)(y − ȳ)
r  =  ───────────────────────────────────
        √[ Σ(x − x̄)² ]  ·  √[ Σ(y − ȳ)² ]


Computational form (easier by hand):

                  n(Σxy) − (Σx)(Σy)
r  =  ──────────────────────────────────────────────
        √[ n(Σx²) − (Σx)² ] · √[ n(Σy²) − (Σy)² ]

Properties

1.  −1 ≤ r ≤ +1
2.  r = +1  perfect positive linear;  r = −1  perfect negative linear;  r = 0  no LINEAR relationship
3.  UNITLESS — changing units (cm → inches, $ → €) does not change r
4.  SYMMETRIC — r(x, y) = r(y, x)
5.  NOT resistant — one outlier can dominate it
6.  Measures LINEAR association only

Interpreting the magnitude

\|r\| Strength
0.00 – 0.19 Very weak / none
0.20 – 0.39 Weak
0.40 – 0.59 Moderate
0.60 – 0.79 Strong
0.80 – 1.00 Very strong

These are conventions, not laws — in physics r = 0.8 is disappointing; in social science it is remarkable.

The Coefficient of Determination

r²  =  the proportion of the variation in y that is explained by
       the linear relationship with x

r = 0.8 → r² = 0.64 → 64% of the variation in y is explained by x; the other 36% is not. Squaring is what turns a correlation into a statement about explained variation, and r² reappears as the headline number of a regression (13-02).

Testing Whether a Correlation Is Significant

A non-zero r in a sample does not prove a non-zero ρ in the population.

H₀: ρ = 0        (no linear correlation in the population)
H₁: ρ ≠ 0

                  ______
                 /n − 2
t  =  r  ·   √  ────────           df  =  n − 2
                 1 − r²

Worked example of the test

r = 0.85 with n = 10:

t = 0.85 × √(8 / (1 − 0.7225)) = 0.85 × √(8 / 0.2775) = 0.85 × √28.83 = 0.85 × 5.369 = 4.564

df = 8,  critical t(0.025, 8) = ±2.306
|4.564| > 2.306  →  REJECT H₀

p-value = 2 × P(T₈ > 4.564) = 0.0018

The correlation is statistically significant at α = 0.05.

Note

With n = 100, an r of just 0.20 is significant (p = 0.046) — and explains only 4% of the variation. Significance and strength are different questions. Always report both r and r².

Correlation Is Not Causation

A strong r between x and y has at least four possible explanations:

  1. x causes y
  2. y causes x (reverse causation)
  3. A confounding variable z causes both
  4. Coincidence — a spurious correlation, especially likely when many pairs are screened

Classic examples: ice-cream sales and drowning deaths (confounder: summer temperature); shoe size and reading ability in children (confounder: age).

Only a randomized experiment (08-01) licenses a causal claim from an association.

Alternatives to Pearson

Coefficient Measures Use when
Pearson r Linear association Both variables quantitative and roughly linear
Spearman ρ Monotonic association (correlation of the ranks) Ordinal data, outliers, or a curved-but-monotonic relationship
Kendall's τ Concordant vs. discordant pairs Small samples, many tied ranks
Point-biserial One binary, one continuous Equivalent to a two-sample t-test
Cramér's V Two categorical variables See 12-02

Spearman is the practical default whenever an outlier or a curve makes Pearson untrustworthy.

Worked Example

Hours studied vs. exam score for 8 students.

Student x hours y score xy x² y²
1 2 55 110 4 3025
2 3 60 180 9 3600
3 5 68 340 25 4624
4 6 72 432 36 5184
5 8 80 640 64 6400
6 9 85 765 81 7225
7 11 88 968 121 7744
8 12 94 1128 144 8836
Σ 56 602 4563 484 46638 
n = 8,  Σx = 56,  Σy = 602,  Σxy = 4563,  Σx² = 484,  Σy² = 46638

Numerator   = n(Σxy) − (Σx)(Σy) = 8(4563) − (56)(602) = 36504 − 33712 = 2792

Denominator = √[8(484) − 56²] · √[8(46638) − 602²]
            = √[3872 − 3136]  · √[373104 − 362404]
            = √736            · √10700
            = 27.1293         × 103.4408
            = 2806.28

r = 2792 / 2806.28 = 0.99491

r  = 0.995        very strong positive linear relationship
r² = 0.990        99.0% of the variation in exam score is explained
                  by the linear relationship with hours studied

Significance:  t = 0.99491 × √(6 / (1 − 0.98984)) = 0.99491 × √590.55
                 = 0.99491 × 24.3013 = 24.18       df = 6
               p-value = 2 × P(T₆ > 24.18) ≈ 0.0000003   →  highly significant

Excel

' ══ CORRELATION ═════════════════════════════════════════════════════
' Hours in A2:A9, Scores in B2:B9
=CORREL(A2:A9, B2:B9)               ' Pearson r      -> 0.99491
=PEARSON(A2:A9, B2:B9)              ' identical
=RSQ(A2:A9, B2:B9)                  ' r²             -> 0.98984
=CORREL(A2:A9,B2:B9)^2              ' same

' Built from the computational formula
=COUNT(A2:A9)                       ' n              -> 8
=SUM(A2:A9)                         ' Σx             -> 56
=SUM(B2:B9)                         ' Σy             -> 602
=SUMPRODUCT(A2:A9, B2:B9)           ' Σxy            -> 4563
=SUMSQ(A2:A9)                       ' Σx²            -> 484
=SUMSQ(B2:B9)                       ' Σy²            -> 46638
=(D1*D4-D2*D3)/(SQRT(D1*D5-D2^2)*SQRT(D1*D6-D3^2))    ' r

' Covariance (correlation's un-standardized cousin)
=COVARIANCE.S(A2:A9, B2:B9)         ' sample covariance
=COVARIANCE.P(A2:A9, B2:B9)         ' population covariance
=COVARIANCE.S(A2:A9,B2:B9)/(STDEV.S(A2:A9)*STDEV.S(B2:B9))   ' = r

' ══ SIGNIFICANCE TEST ═══════════════════════════════════════════════
=CORREL(A2:A9,B2:B9)*SQRT((COUNT(A2:A9)-2)/(1-CORREL(A2:A9,B2:B9)^2))  ' t
=T.DIST.2T(ABS(D10), COUNT(A2:A9)-2)          ' two-tailed p-value
=T.INV.2T(0.05, COUNT(A2:A9)-2)               ' critical t -> 2.44691

' ══ SPEARMAN (no built-in — correlate the RANKS) ════════════════════
=RANK.AVG(A2, $A$2:$A$9, 1)         ' C2: rank of x, fill down
=RANK.AVG(B2, $B$2:$B$9, 1)         ' D2: rank of y, fill down
=CORREL(C2:C9, D2:D9)               ' Spearman's rho

' ══ SCATTERPLOT ═════════════════════════════════════════════════════
' Select A1:B9 ▸ Insert ▸ Scatter (Markers only)
' Right-click a point ▸ Add Trendline ▸ Linear
'   ▸ tick "Display Equation on chart" and "Display R-squared value"

' ══ CORRELATION MATRIX (several variables at once) ══════════════════
' Data ▸ Data Analysis ▸ Correlation
'   Input Range = the whole numeric block, Grouped By: Columns, tick Labels

R

hours  <- c(2, 3, 5, 6, 8, 9, 11, 12)
scores <- c(55, 60, 68, 72, 80, 85, 88, 94)

# ══ ALWAYS PLOT FIRST ═══════════════════════════════════════════════
plot(hours, scores, pch = 19, col = "#5B2A86", cex = 1.4,
     xlab = "Hours studied", ylab = "Exam score",
     main = "Study time vs. exam score")
abline(lm(scores ~ hours), col = "#0FA3A3", lwd = 2)

# ══ CORRELATION ═════════════════════════════════════════════════════
cor(hours, scores)                       # Pearson  -> 0.9949
cor(hours, scores)^2                     # r²       -> 0.9898
cov(hours, scores)                       # covariance

cor(hours, scores, method = "spearman")  # rank correlation
cor(hours, scores, method = "kendall")

# ══ SIGNIFICANCE TEST ═══════════════════════════════════════════════
cor.test(hours, scores)
#  t = 24.18, df = 6, p-value = 3.3e-07
#  95 percent confidence interval: 0.9697 0.9991
#  sample estimate: cor 0.9949

cor.test(hours, scores, method = "spearman")
cor.test(hours, scores, alternative = "greater")     # one-sided

# By hand
r <- cor(hours, scores); n <- length(hours)
t <- r * sqrt((n - 2) / (1 - r^2)); t                # 24.18
2 * pt(abs(t), n - 2, lower.tail = FALSE)            # p-value
qt(0.975, n - 2)                                     # critical t -> 2.4469

# ══ CORRELATION MATRIX ══════════════════════════════════════════════
df <- data.frame(hours, scores, sleep = c(6,7,6,8,7,8,7,9))
round(cor(df), 3)
# library(psych);  corr.test(df)         # matrix of r AND p-values
# library(corrplot); corrplot(cor(df), method = "color")

pairs(df, pch = 19, col = "#5B2A86")     # scatterplot matrix

# ══ WHY YOU PLOT FIRST — Anscombe's quartet ═════════════════════════
data(anscombe)
sapply(1:4, function(i) cor(anscombe[[paste0("x", i)]],
                           anscombe[[paste0("y", i)]]))
# 0.8164 0.8164 0.8164 0.8164   — identical r, four completely different pictures

par(mfrow = c(2, 2))
for (i in 1:4) {
  x <- anscombe[[paste0("x", i)]]; y <- anscombe[[paste0("y", i)]]
  plot(x, y, pch = 19, col = "#5B2A86", main = paste("Set", i))
  abline(lm(y ~ x), col = "#0FA3A3", lwd = 2)
}
par(mfrow = c(1, 1))

# ══ HOW MUCH ONE OUTLIER MATTERS ════════════════════════════════════
h2 <- c(hours, 3); s2 <- c(scores, 95)
c(without = cor(hours, scores), with_outlier = cor(h2, s2))

Python

import numpy as np
import pandas as pd
from scipy import stats
import matplotlib.pyplot as plt

hours  = np.array([2, 3, 5, 6, 8, 9, 11, 12])
scores = np.array([55, 60, 68, 72, 80, 85, 88, 94])

# ══ ALWAYS PLOT FIRST ═══════════════════════════════════════════════
fig, ax = plt.subplots()
ax.scatter(hours, scores, s=70, color="#5B2A86")
m, b = np.polyfit(hours, scores, 1)
ax.plot(hours, m * hours + b, color="#0FA3A3", lw=2)
ax.set(xlabel="Hours studied", ylabel="Exam score",
       title="Study time vs. exam score")

# ══ CORRELATION ═════════════════════════════════════════════════════
np.corrcoef(hours, scores)[0, 1]          # 0.99491
pd.Series(hours).corr(pd.Series(scores))  # same
np.cov(hours, scores, ddof=1)[0, 1]       # covariance

# ══ SIGNIFICANCE TEST ═══════════════════════════════════════════════
res = stats.pearsonr(hours, scores)
res.statistic, res.pvalue                 # 0.99491, 3.3e-07
res.confidence_interval(confidence_level=0.95)

stats.spearmanr(hours, scores)            # rank correlation + p
stats.kendalltau(hours, scores)

# By hand
r, n = res.statistic, hours.size
t = r * np.sqrt((n - 2) / (1 - r**2)); t  # 24.18
2 * stats.t.sf(abs(t), n - 2)             # p-value
stats.t.ppf(0.975, n - 2)                 # critical t -> 2.4469

# ══ CORRELATION MATRIX ══════════════════════════════════════════════
df = pd.DataFrame({"hours": hours, "scores": scores,
                   "sleep": [6, 7, 6, 8, 7, 8, 7, 9]})
df.corr().round(3)
df.corr(method="spearman").round(3)

pd.plotting.scatter_matrix(df, figsize=(7, 7), color="#5B2A86")

# Heatmap of the matrix
fig, ax = plt.subplots()
im = ax.imshow(df.corr(), cmap="PuOr", vmin=-1, vmax=1)
ax.set_xticks(range(3), df.columns); ax.set_yticks(range(3), df.columns)
fig.colorbar(im)

# ══ HOW MUCH ONE OUTLIER MATTERS ════════════════════════════════════
h2 = np.append(hours, 3); s2 = np.append(scores, 95)
np.corrcoef(hours, scores)[0, 1], np.corrcoef(h2, s2)[0, 1]

plt.show()

Quick Reference

Task Excel R Python
Pearson r CORREL(x, y) cor(x, y) np.corrcoef(x,y)[0,1]
r² RSQ(x, y) cor(x,y)^2 r**2
Covariance COVARIANCE.S(x, y) cov(x, y) np.cov(x,y)[0,1]
Significance test T.DIST.2T(ABS(t), n-2) cor.test(x, y) stats.pearsonr(x, y)
Spearman CORREL of RANK.AVG cor(x,y,method="spearman") stats.spearmanr(x, y)
Kendall — cor(x,y,method="kendall") stats.kendalltau(x, y)
Correlation matrix Data Analysis ▸ Correlation cor(df) df.corr()
Scatterplot Insert ▸ Scatter plot(x, y) ax.scatter(x, y)
Scatterplot matrix — pairs(df) pd.plotting.scatter_matrix
CI for r — cor.test(x,y)$conf.int res.confidence_interval()

Common Mistakes

  • Computing r without looking at the scatterplot. Anscombe's quartet exists precisely to make this point.
  • Reading r = 0 as "no relationship". It means no linear relationship.
  • Claiming causation from correlation.
  • Reporting r = 0.7 as "70% explained". It is r² = 0.49 → 49%.
  • Confusing significance with strength — with a big n, a trivial r is significant.
  • Extrapolating a relationship beyond the observed range of x.
  • Using Pearson on ordinal data or data with a heavy outlier; use Spearman.
  • Computing a correlation on aggregated group means and applying it to individuals (the ecological fallacy).

Exercises: 13-01: Exercises — Correlation

⬅️ Previous: 12-03: One-Way ANOVA ➡️ Next: 13-02: Simple Linear Regression