Linear Regression Calculator

Linear regression draws the straight line that comes closest to a set of points, and gives you an equation you can predict from. It is the most used statistical method there is, partly because it is simple and partly because a surprising number of relationships are close enough to straight over the range that matters.

What the line is

y = mx + b, where m is the slope and b is the intercept

Least squares picks the m and b that make the sum of the squared vertical distances from the points to the line as small as possible. Squaring means large misses count disproportionately, and it means positive and negative errors cannot cancel out.

The formulas:

slope = Σ(x - x̄)(y - ȳ) ÷ Σ(x - x̄)²

intercept = ȳ - slope × x̄

The second line means the fitted line always passes through the point of the two means, which is a useful check on any regression.

Reading the slope and intercept

The slope is the change in y for a one-unit change in x, and it carries units. A regression of house price on floor area with a slope of 2,400 means each additional square metre is associated with $2,400 of price.

The intercept is the predicted y when x is zero, and it is often meaningless on its own. A regression of weight on height gives an intercept for a person of zero height, which is not a real quantity. It is there to position the line, not to be interpreted.

Correlation and r squared

The correlation coefficient r runs from -1 to +1 and measures how tightly the points cluster around a straight line. Values near zero mean no linear relationship, which is not the same as no relationship at all.

Squaring it gives r², the share of the variation in y that the line accounts for. An r² of 0.85 means 85% of the variation is explained by x and 15% comes from somewhere else.

r	r²	Reading
0.99	0.98	Points sit almost exactly on the line
0.90	0.81	Strong, with visible scatter
0.70	0.49	Half the variation still unexplained
0.50	0.25	Weak; the line predicts poorly
0.30	0.09	Barely better than the mean of y

Note how quickly r² falls. A correlation of 0.7 sounds substantial and explains under half the variation, which is why r² is the more honest figure to report.

Why the residual plot matters more than r squared

A high r² does not mean a straight line was the right choice. Plot the residuals, the vertical distances from each point to the line, against x, and the shape tells you what r² cannot.

Residuals scattered evenly around zero with no pattern mean the line is appropriate. A curved band means the true relationship bends and a straight line is missing it. A fan shape, widening as x grows, means the variability itself changes with x, which breaks the assumptions behind the standard errors even though the slope may still be usable.

Anscombe's quartet is the standard demonstration: four datasets with identical slope, intercept and r² of 0.67, of which only one is genuinely linear. One is a clean parabola. One is a perfect line with a single outlier dragging it. Plotting the data takes seconds and catches all of them.

Prediction and extrapolation

Substituting an x value into the equation gives a predicted y. Inside the range of the data this is interpolation and is reasonable. Outside it, the line has never been tested and the prediction rests on the assumption that the relationship continues unchanged.

Extrapolation fails routinely, because most real relationships are only locally straight. A growth curve fitted over three years and extended to thirty produces a figure nobody should act on. This calculator flags when a requested prediction sits outside the data range.

Correlation is not causation

A strong fit says the two variables move together. It says nothing about which causes which, or whether a third variable drives both.

Ice cream sales and drowning deaths correlate strongly across a year. Neither causes the other; both rise with temperature. The regression is perfectly valid as a description and completely wrong as a causal claim. Establishing causation needs an experiment, or an argument that rules out the alternatives.

What the standard errors tell you

The standard error of the estimate is the typical vertical distance between a point and the line, in the units of y. The standard error of the slope says how precisely the slope itself is known, and dividing the slope by it gives a t statistic. A t statistic above roughly 2 for a reasonable sample suggests the slope is distinguishable from zero.

Both narrow as the sample grows and as the x values spread wider, which is why a designed experiment deliberately spaces its x values out rather than clustering them.

Working through the sums

The formula is easier to follow as a table. For five points at x = 1 to 5 with y = 2, 4, 5, 4, 5:

x	y	x - x̄	y - ȳ	product	(x - x̄)²
1	2	-2	-2	4	4
2	4	-1	0	0	1
3	5	0	1	0	0
4	4	1	0	0	1
5	5	2	1	2	4
			Sum	6	10

Slope = 6 ÷ 10 = 0.6. Intercept = 4 - 0.6 × 3 = 2.2. The line is y = 0.6x + 2.2, and it passes through (3, 4), the point of the two means, as every least squares line does.

Outliers and leverage

A single point can dominate a fit, and its influence depends on where it sits as much as on how far off it is. A point far from the mean of x has high leverage: it acts on a long arm and can swing the line substantially. An unusual y value near the middle of the x range pulls the line up or down without rotating it much.

The practical test is to remove the suspect point, refit, and see whether the slope changes materially. If it does, the conclusion rests on one observation, which is worth saying out loud rather than burying. Deleting a point because it is inconvenient is not acceptable; deleting one because it was a recording error, and saying so, is.

When a straight line is the wrong shape

Two common fixes when the residual plot shows a curve. Taking logarithms of y straightens exponential growth, so a fit on log y against x recovers a constant growth rate. Taking logarithms of both variables straightens power relationships, and the slope then gives the exponent directly.

Adding a squared term fits a parabola and handles a relationship that rises then falls. Each of these is still least squares, applied to transformed variables, so all the same output applies. What changes is that the slope now describes the transformed scale, and interpreting it means converting back.

Common mistakes

Judging the fit by r² alone. Plot the residuals.

Extrapolating beyond the data. The line was fitted to a range and holds no authority outside it.

Reading causation into the fit. Two variables moving together may share a third cause.

Letting one outlier set the line. A single distant point can dominate the fit, so check whether removing it changes the answer materially.