5.0 EDGES MATTER⧉

Stare at a photograph and ask where its information actually lives. Not in the smooth stretches — a clear sky, a freshly painted wall, a cheek lit by soft window light. In those regions a pixel is almost perfectly predictable from its neighbours; you could erase it and redraw it without anyone noticing. The interesting content sits at the edges: the silhouette of an object against its background, the line where light gives way to shadow, the fine filigree of hair or leaves. This is no coincidence — it is also exactly what the human visual system is tuned to register. We perceive differences — local contrast — far more sharply than absolute brightness, which is why a grey patch looks light on a dark surround and dark on a light one. So a surprising fraction of all image editing collapses to a single question: where are the edges, and what do you want to do about them? This part is organised around the three very different answers, and the punchline that makes them one part rather than three is simple — where the edges are governs the edit.

The first answer is to reconstruct an image from its edges. If what the eye cares about is the gradient — the local change from pixel to pixel — then edit the gradient field directly and rebuild the image whose gradients best match it. Because an edited field is generally not the gradient of any image, "rebuild" means a least-squares fit, $\min_f \int \lVert \nabla f - \mathbf{v}\rVert^2$, whose solution is the Poisson equation $\nabla^2 f = \operatorname{div}\mathbf{v}$. Paste a region by copying its gradients rather than its pixels, pin the boundary to the destination, and any lighting or color mismatch at the seam simply dissolves. This is gradient-domain editing, and the same one idea — change only the guidance field $\mathbf{v}$ and re-solve — powers seamless cloning, gradient-domain high dynamic range (HDR) compression, and photomontage.

The second answer is to smooth everything except across edges. An ordinary blur is indiscriminate: at a bright-sky / dark-mountain boundary it averages sky into mountain, and after tone compression a bright halo glows along the silhouette. The fix is to weight each neighbour not only by how close it is in space but by how similar it is in value, so the average quietly stops at every edge. This is edge-preserving filtering — the bilateral filter and its whole family (joint/cross bilateral, the bilateral grid, the guided filter, local Laplacian filters) — and it underwrites clean local tone mapping, detail enhancement, and edge-aware denoising.

The third answer is the mirror image of the second. Instead of working hard to avoid edges while smoothing, go looking for edges and cut along them. A join is invisible precisely where the image is already changing, so to splice two photos into a panorama, retarget an image to a new aspect ratio, or loop a video seamlessly, you route the boundary through the least-noticeable path and keep it out of smooth regions where a seam would scream. This is seam optimization — seam carving, intelligent scissors, graph cuts, normalized cuts, and their relatives.

Reconstruct from edges, smooth except across edges, cut along edges: three verbs, one noun. The chapters that follow lean on machinery built earlier in the book — least squares and sparse linear solvers, convolution and the fast Fourier transform (FFT), dynamic programming and graph algorithms — and they keep introducing the same two recurring characters. One is affinity: the color or intensity difference between two pixels, read as how much they belong together. Affinity tells an edge-preserving filter which neighbours to average, and it tells a seam which edges not to sever — the very same quantity, used once to pull pixels together and once to forbid a cut. The other is the energy you choose. Once an edit is posed as an optimization, the optimizer is off the shelf and all the design lives in what you decide to penalize — gradient mismatch for Poisson, label disagreement across edges for graph cuts, path cost for seam carving.

The roadmap runs from reconstruction to cutting, then a chooser. Poisson image editing develops reconstruction from edges: gradient-domain editing as the same sparse least-squares solve as the regression chapter, with seamless cloning as its headline application. Bilateral filtering introduces the affinity idea and the canonical edge-preserving filter, then its fast and learned variants. A cluster of chapters then rounds out that family — the locally adaptive regression kernel (LARK) and non-local means, local Laplacian filters, the guided filter, and edge-preserving smoothing recast as an optimization (colorization), where the affinity becomes a sparse global solve. Seam optimization turns to cutting along edges, unifying seam carving, scissors and snakes, graph cuts, and normalized cuts under one machinery: discrete optimization on a graph, and a closing recap lays the toolbox out as a chooser. Keep affinity and the energy in view and the three families stop looking like separate tricks and start looking like one habit of mind — respect the edges.

Contents of this part

▸ full collapsible outline of this part

5.0 EDGES MATTER🔗⧉

Contents of this part

5.0 EDGES MATTER⧉