3.17 Downsampling and aliasing🔗⧉

A digital image is a sampling of a continuous scene — we record the value on a regular grid and throw away everything in between. Most of the time that is harmless. But when the scene carries detail finer than the grid can resolve, something strange happens, and it is one of the most counter-intuitive facts in imaging. The previous chapter built the Fourier machinery; sampling is the thing that machinery explains most cleanly, because a sampling grid is just a comb of evenly shifted spikes. (We treat the reconstruction / upsampling direction — putting values between samples — in the next chapter, Resampling; here the concern is what goes wrong when we throw samples away.)

3.17.1 Aliasing: a high frequency in disguise🔗⧉

Picture photographing a building's facade — rows of fine windows, or a striped shirt — and shrinking the result. Where the stripes are finer than the output pixels, they do not melt into a uniform gray; they reorganize into bold, wrong, low-frequency bands and swirls that are not in the scene at all: moiré. The same effect spins wagon wheels backward in old films and freezes a helicopter's rotor on video. Intuitively, when a pixel must summarize detail it cannot represent, it ought to return the local average — a neutral gray. Instead, the leftover high frequency gets misread as a low one.

The clean case is a single sine wave (Figure 3.17.1). Sample a high-frequency sine too coarsely — fewer than two samples per cycle — and the samples you collect fall into a pattern indistinguishable from a lower-frequency sine. The original travels in disguise as a different frequency; that impostor frequency is its alias. The damage is not blur (lost detail you could shrug off); it is corruption — energy that lands at the wrong frequency and masquerades as real structure. The same masquerade plays out in 2-D on a real photograph (Figure 3.17.2): a facade's regular window grid is a high spatial frequency, and sampling it too coarsely without a pre-filter folds that grid down into bold, slow moiré bands that are not in the building.

3.17.2 Nyquist and the sampling theorem🔗⧉

The rule that prevents it is the Nyquist–Shannon sampling theorem: to represent a signal faithfully you must sample at more than twice its highest frequency. Equivalently, with a fixed sampling grid there is a top frequency — the Nyquist frequency, half the sampling rate — that the grid can represent; anything above it cannot be stored and, if present, folds (aliases) down onto a lower frequency and contaminates it. There is no recovering from aliasing after the fact: once a high frequency has folded onto a low one, the two are added together and can never be told apart.

3.17.3 Sampling in the frequency domain: spectral replicas🔗⧉

In the frequency domain the mechanism is vivid. Sampling on a grid is multiplying the signal by a comb of spikes, and multiplying by a comb in space replicates the spectrum — it pastes shifted copies of the signal's spectrum at every multiple of the sampling rate (Figure 3.17.3). If the original spectrum is narrow enough (band-limited below Nyquist), the copies sit side by side without touching and the original can be isolated. If it is too wide, the copies overlap, the tails fold back over the genuine low frequencies, and that overlap is aliasing. Seen this way the fix is obvious: band-limit the signal before you sample it. Run a low-pass filter — a blur — first, to delete the frequencies above Nyquist that would otherwise fold, then sample. This pre-filter (anti-aliasing) trades a little sharpness for the removal of false structure, and it is why a well-written image-shrinking routine blurs before it downsamples. The space-versus-frequency tradeoff returns: a perfectly sharp cutoff in frequency is a sinc spread out forever in space, so practical pre-filters compromise.

3.17.4 Seeing it: sampling and pre-filtering, hands on🔗⧉

The cleanest way to feel all of this is to drive it. The first demo is the one-dimensional story — a single signal, your hand on the sampling rate — and the second is the two-dimensional one, the same physics on a real photograph where aliasing shows up as moiré.

3.17.5 The ideal reconstruction filter: sinc, and why it is unreachable🔗⧉

The deepest reconstruction filter, the one Nyquist's theorem secretly invokes, is the sinc: in the frequency domain it is a perfect box — keep everything below Nyquist, kill everything above, with a vertical wall in between. The sinc reconstructs a band-limited signal exactly. But that vertical frequency wall costs a kernel that ripples on forever in space (the space–frequency tradeoff again), and infinitely wide kernels are neither practical nor well-behaved — they ring. So real systems use approximations to the sinc with finite support (Figure 3.17.6), each striking its own bargain between losing a little mid-frequency contrast and letting a little high-frequency aliasing through. We will meet the practical menagerie — bilinear, bicubic, Lanczos — in the resampling chapter; here the point is only that the ideal exists, that it is unreachable, and that every real filter is a compromise away from it.