Two Variables at Once

Every part of this path so far has looked at one variable at a time. Part 3 asked which centre to trust for a single list of numbers. Part 4 asked how spread out that list is. Part 8 asked whether any single value in that list looks out of place. All of that machinery describes one column of data in isolation.

Most interesting questions in data work involve two columns at once. Does study time relate to exam score? Does advertising spend relate to sales? Does a patient's age relate to their recovery time? These are questions about a relationship between two variables, and they need a different kind of tool. This part introduces the two classic tools for that job, covariance and correlation, and then spends a good portion of the time on something just as important: the many ordinary ways a correlation number can mislead someone who trusts it without looking closer.

Look Before You Calculate: The Scatterplot

Before any arithmetic, the most useful habit when comparing two variables is to plot one against the other as points on a grid, one axis per variable. This is called a scatterplot. Each row of data becomes one dot. If the dots drift upward from left to right, the variables tend to rise together, a positive relationship. If they drift downward, one tends to fall as the other rises, a negative relationship. If the dots form a shapeless cloud with no visible direction, there may be no relationship at all, at least not a straight-line one.

Keep this picture in mind through the rest of the article. Covariance and correlation are both attempts to squeeze the pattern in a scatterplot into a single number. A single number is convenient, but it can never show you the full picture, and the second half of this article is really about what gets lost in that squeeze.

Covariance: Do They Move Together

Recall from part 4 that variance measures how far a single variable's values spread from its own mean, by looking at squared deviations from that mean. Covariance asks a similar question about two variables at the same time: when one variable is above its mean, is the other variable usually above its mean too, or usually below it?

The trick is to multiply the deviations together instead of squaring one deviation. For each row of data, take how far x is from its own mean, take how far y is from its own mean, and multiply those two numbers. If x and y tend to be above their means together and below their means together, most of these products will be positive, and the average of the products will be positive. If x tends to be above its mean exactly when y is below its mean, most products will be negative, and the average will be negative. If there is no consistent pattern, positive and negative products will roughly cancel out, and the average will sit near zero.

As with variance in part 4, when working with a sample rather than a full population, the average is taken by dividing by n minus 1 rather than n. The reasoning is the same reasoning covered there: dividing by n slightly underestimates the true spread in the population the sample came from, and n minus 1 corrects for that. Most statistics software uses n minus 1 by default for sample data, and this article does the same.

Here is a small worked example. Five students report hours studied for a test and the score they got, out of 100.

  • Student A: 1 hour studied, score 50
  • Student B: 2 hours studied, score 55
  • Student C: 3 hours studied, score 65
  • Student D: 4 hours studied, score 70
  • Student E: 5 hours studied, score 80

The mean of the hours is 3, and the mean of the scores is 64. Now compute, for each student, how far their hours are from 3, and how far their score is from 64.

  • Student A: hours deviation -2, score deviation -14, product 28
  • Student B: hours deviation -1, score deviation -9, product 9
  • Student C: hours deviation 0, score deviation 1, product 0
  • Student D: hours deviation 1, score deviation 6, product 6
  • Student E: hours deviation 2, score deviation 16, product 32

Add up the five products: 28 + 9 + 0 + 6 + 32 = 75. Divide by n minus 1, which is 4, and the sample covariance is 75 divided by 4, which equals 18.75. The number is positive, which matches the picture: students who studied more hours than average also tended to score higher than average, and students who studied less tended to score lower.

The Trouble with Covariance: It Has Units

A covariance of 18.75 sounds like something, but it is hard to say what. Its units are hours times points, a unit that does not correspond to anything concrete. Worse, the number changes if you simply measure one of the variables differently, even though the underlying relationship between study time and score has not changed at all.

Suppose the hours had been recorded in minutes instead: 60, 120, 180, 240, 300. Every deviation from the mean also gets multiplied by 60, so every product of deviations gets multiplied by 60, and the covariance becomes 60 times 18.75, which is 1125. Nothing about the relationship between studying and scoring has changed. Only the ruler used to measure one variable has changed, and the covariance moved by a factor of 60 regardless. A covariance of 1125 is not a stronger relationship than a covariance of 18.75; it is the same relationship measured with a different unit.

This is the core weakness of covariance: its size depends on the units of both variables, so it cannot be compared across different pairs of variables, and its raw value gives no sense of how strong a relationship is. A number is needed that keeps the sign and the idea of covariance, but removes the units. That number is correlation.

Correlation: Covariance Without the Units

The fix is the same idea used throughout part 4 and part 7: divide by the standard deviation to turn a raw measurement into something unit-free, similar to how a z-score tells you how many standard deviations a value sits from the mean. The Pearson correlation coefficient, usually written r, divides the covariance of x and y by the standard deviation of x multiplied by the standard deviation of y.

For the study hours example, the sample variance of the hours is the average squared deviation, (4 + 1 + 0 + 1 + 4) divided by 4, which is 2.5, so the standard deviation is the square root of 2.5, about 1.58. The sample variance of the scores is (196 + 81 + 1 + 36 + 256) divided by 4, which is 142.5, so the standard deviation is the square root of 142.5, about 11.94.

Correlation is the covariance, 18.75, divided by the product of the two standard deviations, 1.58 times 11.94, which is about 18.87. That gives r of about 0.99. This single number says: across these five students, hours studied and exam score moved together almost perfectly in a straight-line sense.

Now repeat the minutes experiment. With hours measured as minutes, the covariance became 1125, but the standard deviation of the minutes also scaled by 60, to about 94.87. Dividing 1125 by 94.87 times 11.94 gives back the same r, about 0.99. Changing the unit of measurement changed the covariance but left the correlation exactly where it was. That is the whole point of correlation: it is built to ignore the units and the scale, and to only describe the strength and direction of the straight-line relationship.

Correlation always falls between -1 and 1. An r of 1 means a perfect positive straight-line relationship: every point would fall exactly on an upward-sloping line if plotted. An r of -1 means a perfect negative straight-line relationship, every point on a downward-sloping line. An r of 0 means no straight-line relationship at all. Real data almost never lands on exactly -1, 0 or 1; it sits somewhere in between, and judging what counts as a strong or weak value always depends on the field and the question.

  • Roughly 0 to 0.1 in size: little to no linear relationship
  • Roughly 0.1 to 0.3: a weak relationship
  • Roughly 0.3 to 0.5: a moderate relationship
  • Roughly 0.5 to 0.7: a strong relationship
  • Roughly 0.7 to 1: a very strong relationship
  • These bands are rough conventions, not laws. A correlation of 0.3 can be an important finding in one field and a meaningless one in another, depending on what else is known about the data.

In practice this is almost never computed by hand. The code below reproduces the study hours example with numpy and pandas, two of the most common Python libraries for this kind of work, assuming reasonably recent versions such as numpy 1.2x and pandas 2.x.

🐍Python
import numpy as np
import pandas as pd

hours = [1, 2, 3, 4, 5]
scores = [50, 55, 65, 70, 80]

df = pd.DataFrame({"hours": hours, "scores": scores})

# Sample covariance matrix (ddof=1 means divide by n-1, the default for pandas)
print(df.cov())

# Correlation matrix
print(df.corr())

# Equivalent with numpy, note np.cov defaults to ddof=1 as well
print(np.cov(hours, scores))
print(np.corrcoef(hours, scores))

Running this prints a 2 by 2 covariance matrix whose off-diagonal entry is 18.75, matching the hand calculation, and a 2 by 2 correlation matrix whose off-diagonal entry is about 0.993. The diagonal entries of the covariance matrix are just the variances of each column on its own, and the diagonal of the correlation matrix is always 1, since any variable is perfectly correlated with itself.

Why a Single Number Can Mislead You

Correlation is popular precisely because it compresses a whole scatterplot into one number between -1 and 1. That convenience is also its biggest danger. A single number cannot show shape, cannot show outliers, cannot show whether the data really represents the group someone cares about, and cannot say anything about cause and effect. Each of the next sections walks through a concrete way this goes wrong, using small worked examples rather than vague warnings.

It Only Sees Straight Lines

Correlation measures how well a straight line fits the relationship between two variables. It has nothing to say about curved or other nonlinear relationships, even perfectly deterministic ones. Consider these five points, where y is simply x squared.

  • x = -2, y = 4
  • x = -1, y = 1
  • x = 0, y = 0
  • x = 1, y = 1
  • x = 2, y = 4

If you know x, you know y exactly, with no uncertainty at all. This is about as strong a relationship as two variables can have. Yet the mean of x is 0, and the deviations of x are -2, -1, 0, 1, 2. The mean of y is 2, and the deviations of y are 2, -1, -2, -1, 2. Multiply matching deviations: (-2)(2) = -4, (-1)(-1) = 1, (0)(-2) = 0, (1)(-1) = -1, (2)(2) = 4. These sum to exactly 0, so the covariance is 0, and the correlation is 0.

A correlation of exactly 0 here does not mean x and y are unrelated. It means they have no straight-line relationship, because the relationship curves symmetrically: as x moves away from 0 in either direction, y increases either way, and the positive and negative deviations of x cancel out when paired with the deviations of y. The lesson is blunt: a correlation near 0 only rules out a linear pattern. It says nothing about curved, cyclical or other structured patterns, which is exactly why looking at the scatterplot first matters more than looking at r first.

One Point Can Run the Whole Show

Part 8 covered how a single outlier can distort a mean or a standard deviation. Correlation is at least as vulnerable, because it is built from products of deviations, and one very large deviation, multiplied by another very large deviation, can dominate the sum that everything else is added to.

Take this small dataset of five points with no clear relationship.

  • x = 1, y = 2
  • x = 2, y = 1
  • x = 3, y = 3
  • x = 4, y = 2
  • x = 5, y = 1

Working through the same steps as before, the mean of x is 3 and the mean of y is 1.8. The covariance comes out to -0.25, and dividing by the standard deviations of x (about 1.58) and y (about 0.84) gives a correlation of about -0.19, a weak relationship, consistent with the cloud of points having no strong pattern.

Now add one more point, a clear outlier: x = 20, y = 50. With six points instead of five, the mean of x jumps to about 5.83 and the mean of y jumps to about 9.83. The new point's deviation from the mean is enormous in both directions, about 14.17 for x and 40.17 for y, and the product of those two deviations alone is about 569, dwarfing the combined contribution of the other five points, which summed to about 113. Carrying the full calculation through gives a covariance of about 136.3 and a correlation of about 0.98.

One extra data point turned a weak negative correlation of -0.19 into a near-perfect positive correlation of 0.98. Nothing changed about the relationship among the original five points; a single point reshaped the headline number almost entirely on its own. This is exactly the kind of situation part 8 was built to help you catch: before trusting a correlation, check whether the value survives with and without the most extreme points. If it does not, the number describes one unusual case more than it describes the overall relationship, and the right response is to investigate that case, not to quote the correlation at face value.

Correlation Is Not Causation

This is probably the most repeated warning in statistics, and it is repeated because it is routinely ignored. A correlation between two variables tells you they move together. It tells you nothing about why. There are at least four different explanations consistent with any observed correlation between x and y: x causes y, y causes x, some third variable causes both x and y, or the relationship is a coincidence of the particular data collected.

A commonly used illustration involves ice cream sales and drowning incidents, which tend to rise and fall together across the months of the year. Nobody believes buying ice cream causes drowning, or that drowning causes ice cream sales. Both tend to rise in warmer months, when more people swim and more people buy ice cream. The warm weather is a confounding variable, something that influences both of the measured variables and creates a correlation between them even though neither one causes the other.

The same pattern shows up in plenty of less obvious places. Among children, shoe size and reading ability tend to be positively correlated, not because bigger feet help with reading, but because both tend to increase with age. Age is the confounder. Whenever a correlation shows up between two variables, it is worth asking directly what third factor might be moving both of them, especially a factor connected to time, age, size, or location, since these quietly drive a great many pairs of variables in the same direction.

Hidden Groups Can Flip the Sign

A correlation computed on a combined dataset can point in the opposite direction from the correlation within each group that makes up that dataset. This happens often enough to deserve its own worked example, built here with small constructed numbers purely to show the mechanism, not as a claim about any real dataset.

Group A has three points where y decreases as x increases.

  • x = 1, y = 5
  • x = 2, y = 4
  • x = 3, y = 3

Group B also has three points where y decreases as x increases, following the exact same pattern, just shifted to larger values of both variables.

  • x = 11, y = 15
  • x = 12, y = 14
  • x = 13, y = 13

Within Group A, every unit increase in x comes with exactly a one unit decrease in y, so the points fall on a perfect downward line, and the correlation within that group is exactly -1. The same is true within Group B, also exactly -1. Both groups, taken on their own, show a perfect negative relationship.

Now combine the six points into one dataset and compute the correlation across all of them together, ignoring which group each point came from. The mean of x across all six points is 7, and the mean of y is 9. Working through the deviations and products as before gives a covariance of about 29.2, and dividing by the standard deviations of x and y, both about 5.55, gives a correlation of about 0.95. The combined dataset shows a strong positive correlation of 0.95, even though every single group inside it shows a perfect negative correlation of -1.

The reason is that Group B sits at much higher values of both x and y than Group A. The jump between groups is far larger than the gentle downward slope within either group, so when both groups are lumped together, the dominant pattern in the combined data is simply that Group B has higher x and higher y than Group A, producing a positive overall trend that has nothing to do with the real within-group relationship. This is closely related to Simpson's paradox, a well-known phenomenon where a trend present in separate groups reverses or disappears once the groups are combined. The practical lesson is to always ask whether an obvious grouping variable exists, such as department, region, age band, time period or customer segment, and to check the correlation within each group separately before trusting a correlation computed on the whole mixed dataset.

Restricting the Range Hides the Relationship

Correlation also depends heavily on how wide a slice of the data you are looking at. Imagine the true relationship between two variables is strong across their full range, but you only have data from a narrow slice of one of them. The correlation computed on that narrow slice can look much weaker than the true underlying relationship, simply because most of the variation that would reveal the pattern has been cut away.

A standard example is the relationship between an entrance test score and later performance, measured only among people who were admitted because their test scores were already high. Across the full population of everyone who took the test, score and later performance might be strongly related. But among the admitted group alone, nearly everyone already has a similarly high score, so there is very little variation left in the test score to correlate with anything. The correlation computed on just the admitted group can come out much weaker than the true relationship in the wider population, not because the relationship changed, but because the range of one variable was artificially narrowed before the correlation was computed.

A useful way to picture this is to imagine zooming into a thin vertical strip of a scatterplot that otherwise shows a clear upward trend. Inside that one narrow strip, the points might look almost randomly scattered vertically, since the strip only spans a small part of the x-axis. The trend is still there in the full picture; it has just been hidden by only looking at a slice of it. Whenever a correlation is computed on a subgroup that was selected using one of the two variables, such as already-admitted students, approved loan applicants, or surviving companies, this restriction of range is worth checking for before concluding the relationship is weak.

With Many Variables, Some Correlate by Chance

If you compute the correlation between only two variables that you chose for a specific reason, a strong result is meaningful evidence worth investigating further. If instead you compute the correlation between every possible pair among dozens or hundreds of variables and then pick out the highest ones you find, some of those high correlations will be there purely by chance, with no real underlying relationship at all. This is sometimes called the multiple comparisons problem, and it applies to correlation just as it applies to other statistical tests covered later in this learning path.

The mechanism is simple: with a small sample, random fluctuation alone can occasionally produce what looks like a moderately strong correlation between two variables that have nothing to do with each other. If you only test one pair, that chance is small. If you test hundreds of pairs, the chance that at least one of them will look surprisingly correlated purely by luck becomes fairly large. This is why a correlation discovered by scanning a wide dataset for interesting patterns, without a specific reason to expect a relationship beforehand, deserves more scepticism than one that confirms a hypothesis stated in advance. Checking whether the correlation holds up on a fresh batch of data, collected after the pattern was spotted, is one of the more reliable ways to tell a genuine relationship from a coincidence.

A Checklist Before You Trust a Correlation

  • Plot the two variables as a scatterplot before looking at the number. A shape that a single statistic cannot show often changes the entire interpretation.
  • Check whether the relationship looks like a straight line. If it curves, cycles, or clusters, correlation will undersell or misdescribe it.
  • Recompute the correlation with and without the most extreme points. If removing one or two points changes the answer a lot, say so plainly rather than quoting the original figure alone.
  • Ask what third variable could be driving both measured variables at once, especially variables connected to time, age, size or location.
  • Check whether the data covers the full range of interest, or only a narrow slice selected using one of the variables themselves.
  • Look for an obvious grouping variable, such as department, region or time period, and check the correlation within each group separately.
  • Remember that a strong correlation shows association, never proof of cause and effect, on its own.
  • Treat correlations discovered by scanning many variable pairs with more caution than correlations that confirm a specific, pre-stated expectation.

Covariance vs Correlation: Picking the Right Tool

Given that correlation fixes the units problem, it is natural to ask why covariance is still used at all. The short answer is that covariance is often the more useful building block inside further calculations, even though it is a poor number to report on its own. Many techniques used further along a typical data science path, such as combining the risk of multiple financial assets or reducing the number of variables in a dataset, are built directly on covariance, because the raw scale of the variables matters to those calculations in a way it does not matter when the goal is simply to describe how strongly two variables move together.

For everyday description and communication, correlation is almost always the better choice, precisely because it is bounded between -1 and 1 and does not depend on the units chosen for either variable. When the goal is to report, compare, or explain a relationship between two variables to another person, correlation is the number to reach for, with all the caveats from this article attached to it.

Summary and What's Next

Covariance measures whether two variables move together by averaging the product of their deviations from their own means, but its value depends on the units of measurement, which makes it hard to interpret or compare on its own. Correlation fixes this by dividing covariance by the standard deviations of both variables, producing a number always between -1 and 1 that is unaffected by changes in scale or units.

A correlation coefficient is useful, but it is a heavy compression of a scatterplot into a single number, and that compression throws away information that matters. It only captures straight-line relationships and can sit at zero even for a perfectly predictable curved relationship. A single outlier can swing it from near zero to near one. Groups hidden inside a combined dataset can make it point in the opposite direction from the pattern within each group. Looking only at a narrow slice of one variable can make a real relationship look weak or absent. Scanning many variables for high correlations will turn up some by chance alone. And even a strong, reliable correlation never on its own tells you which variable, if either, is causing the other to move.

The next part in this path, Populations, Samples and Sampling Bias, looks at a question that sits underneath everything covered so far: the numbers calculated in this path, means, spreads, correlations, are almost always computed from a sample rather than from every possible data point, and the way that sample was collected can shape every statistic built from it, correlation very much included.