10.6 Guided image filtering🔗⧉

A clean guide can help smooth a noisy target or refine a low-resolution mask. The guide must contain relevant, aligned structure: a strong texture edge is not necessarily an object or depth boundary.

The guided filter fits a regularized affine relation between guide and target in overlapping windows He et al. 2010. Local statistics and coefficient averaging give an efficient implementation using box sums.

Imagine fitting the input against the guide values in a small window. If the guide varies and those variations predict the input, a fitted slope transfers some of that structure into the output. If the guide is almost constant, the regularized fit suppresses the slope and the local mean becomes more important. A guide edge alone is therefore insufficient to guarantee an output edge: the fitted relationship and regularization determine how strongly it is transferred.

Overlapping windows give multiple predictions for each pixel. Averaging their fitted coefficients combines those predictions. The required local means and correlations can be computed by box filters, which is the source of the efficient implementation. The method replaces repeated range-weight evaluation with local regression statistics; it does not promise immunity to every edge artifact.

10.6.1 The key idea: the output is a local line in the guide🔗⧉

Here is the whole filter in one picture. Take a small window of the image, say a $9\times9$ patch. Inside it, look at the guidance image $I$ (the image whose edges we trust) and the input $p$ (the possibly-noisy image we want to clean). Plot, for every pixel in the window, the guide value $I$ against the input value $p$, one dot per pixel. Now fit a straight line through that cloud of dots: find the slope $a$ and intercept $b$ for which

best matches the input over the window, in the least-squares sense. The fitted line is the filter: the output $q$ at each pixel is whatever the line predicts from that pixel's guide value, $q = aI + b$.

Read this back in words: in each little window, explain the output as the best linear function of the guidance image. Two consequences follow immediately, and they are the entire reason the filter works.

Within one fitted window, a guide step produces an output step scaled by $a$. If the target does not covary with the guide, $a$ may be zero or negative. The completed filter averages overlapping fits, so the one-window relation alone does not prove preservation of every edge.

Second, flat regions get smoothed. Where the guide is nearly constant over the window, there is no edge for the line to lock onto; the slope $a$ collapses toward zero and the output becomes nearly $q \approx b$, a local average of the input. So in the interior of a smooth region the filter behaves like a plain blur (exactly what we want) while at the guide's edges it holds the line. One model, two behaviors, selected automatically by whether the guide has a gradient to track.

10.6.2 Fitting the line, and the one knob🔗⧉

What sets the slope $a$? Ridge-regularized least squares over the window gives: the slope is the covariance of guide and input divided by the variance of the guide:

where the subscript $w$ means "averaged over the window," $\bar I_w$ and $\bar p_w$ are the window means, and $\varepsilon$ is a regularization parameter. The intercept $b$ is then just whatever makes the line pass through the window's center of mass $(\bar I_w, \bar p_w)$. Read the slope back: how strongly does the input track the guide here, relative to how much the guide itself varies? Where guide and input rise and fall together (an edge they share), the covariance is large, the slope is steep, and the edge is carried through. Where the guide is flat, its variance is tiny and, but for $\varepsilon$, the slope would be ill-defined; $\varepsilon$ is what keeps it tame.

The regularizer penalizes large slopes. It has units of guide intensity squared: rescaling the guide by a factor requires rescaling $\varepsilon$ by its square for equivalent behavior. The window radius is another important control. Larger windows increase the region sharing a fit.

One loose end: windows overlap. A given pixel lies inside many windows (one centered on each of its neighbors) and each of those windows fits its own line, so each makes its own prediction for that pixel. The guided filter resolves this in the simplest possible way: it averages the per-window coefficients. Compute $(a,b)$ for every window, then for each pixel set $\bar a$ and $\bar b$ to the average of the $a$'s and $b$'s of all windows covering it, and output

At a fixed pixel, averaging predictions $a_k I_i+b_k$ over its windows equals averaging the coefficients first. The guide value $I_i$ is common to those predictions. The coefficients vary spatially, so the exact derivative includes additional terms: $\nabla q=\bar a\nabla I+I\nabla\bar a+\nabla\bar b$.

In a nearly constant guide window, fitting a large slope does little to explain variation and the regularizer pulls the slope toward zero. The prediction then approaches a local constant. Where guide and input covary strongly, the fit can retain a larger slope and carry structure from the guide into the output. Guide variance alone is not a semantic edge detector: covariance with the filtered input matters too. This is why an unrelated edge in a guide can be unhelpful, and why the guide's scale and the regularization units must be kept together.

10.6.3 Why it is fast: O(N) regardless of window size🔗⧉

The bilateral's cost is its main drawback: a brute-force bilateral over a window of radius $r$ does work proportional to the window area at every pixel, so widening the window makes it slower. The guided filter has no such dependence. Look back at the recipe: every quantity it needs (the means $\bar I_w, \bar p_w$, the variance, the covariance) is a local average over the window, and a local average is exactly a box filter. And a box filter, computed with a running sum (a sliding accumulator that adds the pixel entering the window and subtracts the one leaving), costs the same per pixel no matter how big the window is. (The variance and covariance need only the means of $I$, $p$, $I^2$, and $Ip$, each a box filter, combined pointwise, for instance $\operatorname{var}_w(I) = \overline{I^2}_w - \bar I_w^{\,2}$.)

So the entire filter is a fixed number of box filters plus some pointwise arithmetic, and its total cost is O(N), linear in the number of pixels $N$, with a constant that does not grow with the window radius. This is the headline practical advantage. Where the bilateral forces a trade between window size and speed, the guided filter charges the same whether the window is $3\times3$ or $33\times33$, which is precisely why it became the go-to fast edge-preserving filter and a standard drop-in replacement for the bilateral when speed matters.

10.6.4 No gradient reversal🔗⧉

The bilateral has a second, subtler failure. Gradient reversal is an artifact in which a smooth ramp, after filtering, comes back with its slope locally reversed: a little overshoot or undershoot that creates a faint false edge where the input had none. It happens near strong edges, and its cause is the bilateral's range weighting: close to an edge, a pixel's neighborhood is dominated by samples from one side, the weighted average pulls toward that side, and the reconstructed detail can tip the wrong way. Sharpening or detail-boosting on top of a bilateral base layer makes these reversals plainly visible as visible rings and ridges.

The original paper demonstrates reduced gradient-reversal artifacts in detail enhancement He et al. 2010. The local affine model explains that behavior, but does not prove global monotonicity for arbitrary guides and inputs. Cross-guided slopes can be negative, and the spatially varying coefficients contribute their own derivatives. Test aggressive enhancement rather than promising that overshoot is impossible.

10.6.5 Reading the affinity back out🔗⧉

We have not mentioned an affinity explicitly, with no range weight, patch distance, or smoothness energy, and yet the guided filter belongs squarely in the affinity family. To see it, ask the implicit question: in computing the output at pixel $i$, how much does the input at some other pixel $j$ contribute? Expand the algebra (substitute the line fit into the coefficient average and collect terms) and the guided filter turns out to be, like every method in this chapter, a weighted average of the input, $q_i = \sum_j W_{ij}\,p_j$, with a specific weight kernel

The kernel collects contributions from every window containing both pixels. Similar deviations from a window mean often increase their coupling; opposite deviations can reduce it or make it negative. This explains a relation to image affinity without identifying the weights with a bilateral Gaussian.

For a fixed guide the output is linear in the target, but the effective weights can be negative. They should not be interpreted as probabilities or guaranteed convex-averaging weights. For RGB guidance, replace the scalar variance division by a regularized 3×3 covariance solve.

mean_I = box_mean(I); mean_p = box_mean(p)
variance = box_mean(I*I) - mean_I*mean_I
covariance = box_mean(I*p) - mean_I*mean_p
a = covariance / (variance + epsilon)
b = mean_p - a*mean_I
q = box_mean(a)*I + box_mean(b)

Use the same window and boundary convention in all box means. Accumulation precision matters when subtracting nearly equal moments.

10.6.6 Where it is used🔗⧉

Box-filter evaluation makes this local model useful in several tasks:

• Smoothing and detail control: use the input as its own guide, then inspect the base and residual.
• Guided denoising or upsampling: use registered guide structure, but reject mismatched shadows, textures and occlusions.
• Mask or transmission refinement: guide a coarse estimate with image structure. This does not replace the physical or semantic model that produced the estimate.
• Differentiable processing: with positive regularization and fixed window conventions, the arithmetic can be incorporated into a learned pipeline.

Dehazing provides a concrete reason for this choice. A dark-channel method first estimates a coarse transmission map, then refines it against the photograph before recovering scene radiance (see Chapter 24.3, Dehazing). The original refinement used a matting-Laplacian solve; guided filtering offers a faster local alternative for that refinement. This is not a new atmospheric model. It changes how an existing estimate is made to follow image structure.

The same separation is useful for depth, chrominance or a coarse network prediction. Compute the expensive estimate at low resolution, then use the full-resolution image to guide its refinement. The guide can supply boundary locations that were lost in downsampling, but cannot establish that every texture edge belongs in the estimated quantity. Mask feathering and flash/no-flash denoising use the same mechanism with different targets and different risks of guide mismatch.