The Gaussian, taken apart

The probability density of the univariate standard Gaussian (also called the standard normal) is

\phi(x)= \frac{1}{\sqrt{2\pi}}IVnormalizing constant e^{-x^2/2}Ikernel

Shift and scale

If Z \sim \mathcal{N}(0, 1), then X = \mu + \sigma Z \sim \mathcal{N}(\mu, \sigma^2), so every Gaussian is a shifted and scaled standard Gaussian.

Compare with

From density to probability

For a continuous random variable, probability is area under the density, P(a < X < b) = \int_a^b f(x)\,dx.

The density f(x) of X \sim \mathcal{N}(0, \sigma^2)
The CDF F(x) = P(X \le x), the area to the left of x

z-scores

The z-score of a value x is z = (x - \mu)/\sigma, the number of standard deviations x lies above or below the mean. If X \sim \mathcal{N}(\mu, \sigma^2), then (X - \mu)/\sigma \sim \mathcal{N}(0, 1). The heights below are of US women and men, each group modeled as a Gaussian.

A woman
percentilenormal modelNHANES sample
A man
percentilenormal modelNHANES sample
One-tail probability P(Z > |z|), as 1 in N

Where does π come from?

The kernel e^{-x^2/2} has no closed form antiderivative, so the area A under it can't be found directly. The two dimensional Gaussian looks harder but is actually easier to work with, and A can be recovered from its volume. The π in the normalizing constant emerges because the 2D Gaussian is constructed from circles.

drag to turn

A is the area under the kernel, A = \int_{-\infty}^{\infty} e^{-x^2/2}\,dx. Every slice of f is that kernel times a constant, so A shows up in its area. The slice areas add up to 1, and that pins down A.

Where the draws land

Each dot is one draw from the standard Gaussian in one, two or three dimensions. Below each plot is the distribution of a draw's distance from the center.

One dimension
f(x) = \frac{1}{\sqrt{2\pi}}\,e^{-x^2/2}
R = |X|, with density \sqrt{2/\pi}\, e^{-r^2/2}
Two dimensions
f(x, y) = \frac{1}{2\pi}\,e^{-(x^2+y^2)/2}
R = \sqrt{X^2 + Y^2}, with density r\, e^{-r^2/2}
Three dimensions
f(x, y, z) = \frac{1}{(2\pi)^{3/2}}\,e^{-(x^2+y^2+z^2)/2}
R = \sqrt{X^2 + Y^2 + Z^2}, with density \sqrt{2/\pi}\, r^2 e^{-r^2/2}

Round and square: only the Gaussian

Round means rotationally invariant, which happens exactly when f depends only on x^2 + y^2. Square means X and Y are independent with the same distribution, f(x, y) = g(x)\,g(y).

drag to turn
Do the marginals multiply to the joint density?
Conditional densities of X given Y = 0, 1, 2

The central limit theorem

Add up independentWe can even relax this assumption and allow for weak dependence, like rainy days that come in streaks. The sum still ends up close to normal, though with a different standard deviation than independence would give., identically distributedWe can even relax this assumption and allow the X_i to have different distributions, as long as no single one dominates the sum. random variables X_1, X_2, \ldots, X_n and you get a new random variable S_n with its own distribution. What happens to its shape as n grows? Does it depend on the distribution of the components?

Each component X_i
Die
Coin
Data
Plot
largest errorThe largest difference, over all s, between the exact P(S_n \le s) for the sum S_n of n components and its normal approximation \Phi((s - n\mu)/(\sigma\sqrt{n})), where \Phi is the standard normal CDF. By the Central Limit Theorem, it goes to 0 as n grows.

Each outcome \omega = (x_1, \ldots, x_n) has probability p(\omega) = p(x_1) \times \cdots \times p(x_n), so the pmf of S_n is P(S_n = k) = \sum_{\omega : S_n(\omega) = k} p(\omega).

Components
Event