A Gaussian world

One of the most fun I had during my undergrad years was learning how to extract a clean signal from white noise, and the mathematical beauty of how it's done. It's one of the reasons I really enjoyed studying signals and systems.

Every signal lives in two places at once. There's the version you can see and hear — voltage over time, pressure over time, whatever the physical quantity is - and there's a frequency-domain version, which describes the same signal as a sum of pure sine waves at different frequencies. The Fourier transform is how you move between the two. A pure tone — say, A4 at 440 Hz — looks like a smooth sine in time, and like two spikes in frequency:

Two spikes. Infinitely tall, infinitely thin, sitting at +f and −f. Those are Dirac deltas. The first time you see this, you do a double take. All the information in the sine wave — its period, its phase, the fact that it does this forever — collapses into two points on a line. A pure sinusoid turns out to be exactly as complicated as the frequencies it contains, which for a single tone is: not very.

Real signals never look this clean. A real sine is the clean one with noise piled on top of it:

The job of a filter is to recover the clean wave from the noisy one. This is where the Fourier transform earns its rent. Gaussian white noise, which is the kind nature throws at you for free, more or less the floor that every physical measurement sits on — has a spectrum that's roughly flat across all frequencies. The signal's spectrum is two tall spikes. In the time domain, signal and noise are tangled together. In the frequency domain, they aren't even close to each other. That asymmetry is what you exploit.

If you can build an operation that flattens the spread-out noise while leaving the spikes alone, you've recovered the signal. In the frequency domain, "flatten" means multiply: take the noisy spectrum and multiply it pointwise by something that's tall in the middle and short at the edges. The spikes survive, the noise floor gets crushed.

The function you multiply by is the filter, and the operation that corresponds to it in the time domain is convolution:

$$(f * g)(x) = \int_{-\infty}^{\infty} f(\tau)\, g(x - \tau)\, d\tau$$

Convolution in time and multiplication in frequency are the same operation seen from two sides. This is the convolution theorem, and it's the reason filtering is something you can actually do on a computer instead of just talk about. You never have to leave the time domain. You just convolve.

So: which function do you convolve with? There's a catalogue — box filters, triangle filters, Butterworths, Chebyshevs, sinc, wavelets. Pick your trade-off, you'll find one that fits. But one filter is special in a way that none of the others are, and that's the Gaussian:

$$g(x) = e^{-x^2/2\sigma^2}$$

A bell curve. Width controlled by σ — small σ means narrow and tall, large σ means wide and flat. What makes it special isn't the shape; many filters look bell-ish. It's what happens when you Fourier-transform it. Almost every other filter shape transforms into something that looks nothing like itself. The Gaussian transforms into another Gaussian.

Here's the proof. Plug $f(x) = e^{-x^2/2\sigma^2}$ into the Fourier transform:

$$\hat{f}(\xi) = \int_{-\infty}^{\infty} e^{-x^2/2\sigma^2}\, e^{-2\pi i \xi x}\, dx$$

The two exponentials combine into one. The exponent is a quadratic in $x$ with a complex linear term, and the trick is complete the square. Yes, that complete the square — it never really leaves. After a little algebra:

$$-\frac{x^2}{2\sigma^2} - 2\pi i \xi x \;=\; -\frac{(x + 2\pi i \xi \sigma^2)^2}{2\sigma^2} \;-\; 2\pi^2 \xi^2 \sigma^2$$

The second term doesn't contain $x$, so it pulls out of the integral:

$$\hat{f}(\xi) = e^{-2\pi^2 \xi^2 \sigma^2} \int_{-\infty}^{\infty} e^{-(x + 2\pi i \xi \sigma^2)^2 / 2\sigma^2}\, dx$$

The integrand is a Gaussian, but shifted along the imaginary axis by an amount that depends on $\xi$. Substitute $u = x + 2\pi i \xi \sigma^2$ and the integral becomes a Gaussian integrated along a horizontal line in the complex plane, parallel to the real axis but offset upward. Whether you can slide it back down is the question that hangs in the air. You can: the integrand has no poles anywhere (it's entire), and the vertical sides of the connecting rectangle decay like $e^{-R^2}$ at infinity, so Cauchy's theorem hands you the contour shift for free. This step gets glossed over in every signals class I ever took, and then properly justified in a complex analysis class three semesters later, by which point most people have stopped asking.

What remains is the standard Gaussian integral:

$$\int_{-\infty}^{\infty} e^{-u^2/2\sigma^2}\, du = \sigma\sqrt{2\pi}$$

A constant. Just a number. Combine everything:

$$\boxed{\hat{f}(\xi) = \sigma\sqrt{2\pi}\; e^{-2\pi^2 \sigma^2 \xi^2}}$$

A Gaussian, with new width $\hat\sigma = 1/(2\pi\sigma)$. Same family, different scale.

Look at the width relationship: $\sigma \cdot \hat\sigma = 1/(2\pi)$. The product is fixed. Narrow in time means wide in frequency and vice versa, with no exit. This is the Heisenberg uncertainty principle, written for signals — same uncertainty, same lower bound, same math. And the Gaussian is the unique function (up to scaling) that hits the lower bound exactly. Every other filter shape is sitting at some non-optimal point on the trade-off curve. The Gaussian is the trade-off curve.

So when you reach for a Gaussian to filter noise out of a signal, you're not just picking something smooth-looking. You're using the one function the math has nominated as optimal — the function whose time and frequency widths balance at the theoretical minimum.

Or, to come back to where I started: sometimes the universe agrees with you that things should be clean.