E
edgecase
quant prep
track

Stats

OLS and time-series intuition. Watch the estimators converge as you change inputs.

OLS — true vs estimated

trueOLS fit

β̂

0.919

true 1.00

α̂

-0.015

true 0.00

R²

0.884

Regression inputs

1.00
0.00
1.00
50

AR(1) process · x_t = φ·x_(t-1) + ε

ACF(1)

0.803

true 0.80

ACF(5)

0.357

true 0.33

ACF(10)

0.113

true 0.11

0.80

φ → 1: random walk (no mean reversion). φ ≈ 0: white noise. Negative φ: oscillating.

What an interviewer expects you to know

OLS — what it actually optimizes

β̂ minimizes the SUM OF SQUARED RESIDUALS. Geometrically it projects y onto the column space of X. The normal equations β̂ = (X'X)⁻¹X'y come from setting the gradient of the SSR to zero. Squared loss isn't a moral position — it's just convenient (smooth, gives the conditional mean under Gaussian noise).

Why R² lies — adjusted R² fixes it

R² always rises (never falls) when you add a regressor, even pure noise. Adjusted R² = 1 − (1 − R²)·(n − 1)/(n − k − 1) penalizes you for spending degrees of freedom. Production research code reports both; the gap reveals over-fitting.

Stationarity — the time-series prerequisite

A series is (weakly) stationary if mean and autocovariance don't depend on absolute time. Non-stationary series (random walks, trending prices) can give SPURIOUS regression results — two unrelated random walks regressed against each other show 'significance' a third of the time. The fix: first-difference, or test for cointegration explicitly.

Autocorrelation breaks OLS standard errors

Standard OLS assumes errors are i.i.d. If residuals are autocorrelated (very common in financial time series), your standard errors are too small and t-stats too big — you see significance where there is none. Newey-West (HAC) standard errors are the standard fix.

AR(1) — the simplest model of mean reversion

x_t = φ·x_{t-1} + ε_t. |φ| < 1: stationary, mean-reverts to 0. φ = 1: random walk, non-stationary, no reversion. φ slightly less than 1: 'almost-random-walk' — looks like a trend over short windows but reverts over long. Most equity spreads in pairs trading fit this regime.

Cointegration in 30 seconds

Two non-stationary series can have a LINEAR COMBINATION that IS stationary. If price_A and price_B are random walks but (price_A − β·price_B) mean-reverts, they're cointegrated — that's the canonical pairs-trade setup. Tests: Engle-Granger (two-step), Johansen (multi-asset).