BookNotesSlidesP-sets Syllabusdraft · v0.2.49.1
◀▲⌂▶
💬Comments welcome. To leave a note, select any text and click the note / highlight button that pops up — or open the panel with the tab at the top-right (‹). Notes are visible only inside our private review group.

18.4 Exposure fusion⧉

For a well-aligned bracket, exposure fusion can produce a finished picture without first fitting a camera response or estimating radiance.

Exposure fusion deserves more attention than its place among the local operators suggests, because it is the one that ships. Fusing a bracket directly, with weights for contrast, saturation and good exposure blended through a Laplacian pyramid, skips the radiance map, the calibration of the camera response and the tone mapping altogether. For the consumer case, the standard factorization of capture, assemble a radiance map, then tone-map it, was the wrong one, and the field spent a decade on a problem that could be sidestepped. Phone pipelines kept the idea: HDR+ renders its merged burst by fusing synthetic exposures made from it (Chapter 18.5, Local Laplacian filtering and other multiscale methods, Chapter 17.6, HDR+ and burst imaging on phones).

Mertens and colleagues weight rendered frames using local contrast, channel variation and proximity to midrange values, then blend them through pyramids (Mertens et al. (exposure fusion)). The frames need a consistent color encoding and scaling across the stack. The method avoids explicit radiometric calibration; it is not invariant to arbitrary encoding changes.

No radiometric calibration is performed and no HDR radiance map is built: the bracket goes in and a rendered picture comes out. Fusion can also combine differently rendered versions of one radiance estimate, which is a rendering step rather than the raw align-and-merge of Chapter 17.3, Merging exposures.

fig-exposure-fusion-1
Figure 18.4.1. Exposure fusion, from the original paper. A bracket of differently-exposed frames and the single fused result, alongside the per-frame quality weights (well-exposedness, contrast, saturation) that steer the multiscale blend: no HDR radiance map is ever built. Reproduced from Mertens, Kautz & Van Reeth (2007) with permission.

18.4.1 Three qualities, one weight⧉

Each pixel in each frame gets a weight from three measures, multiplied together.

Contrast is measured by the magnitude of a Laplacian response. It favors edges, including noise and sharpening artifacts. Flat regions can be correctly exposed even when this measure is small.

Saturation is approximated by standard deviation across RGB channels. Neutral gray also has zero channel variation, so this term is an appearance preference rather than a reliable clipping detector.

Well-exposedness favors values near the middle of the normalized encoded range. A common measure multiplies Gaussian preferences across channels:

$$E_i(\mathbf p)=\prod_{c\in\{R,G,B\}}\exp\!\left[-\frac{(I_{ic}(\mathbf p)-0.5)^2}{2s^2}\right].$$

The width $s$ controls the preference. Midrange code value is not necessarily middle scene luminance.

$$ w_{i}(\mathbf p) = C_i(\mathbf p)^{\alpha}\, \cdot\, S_i(\mathbf p)^{\beta}\, \cdot\, E_i(\mathbf p)^{\gamma} $$

We normalize the weights across the stack so they sum to one at each pixel. Every one of the three measures is computable from the frame alone, with no knowledge of its exposure time.

18.4.2 Blending without seams⧉

Abrupt changes in a weight map can create seams in a direct weighted average. Smooth multiscale blending reduces these transitions, although mismatched geometry, brightness and motion can still remain visible.

We build Laplacian pyramids of the images and Gaussian pyramids of the normalized weights, blend corresponding levels and reconstruct. Use consistent boundaries and pyramid sizes. Fine bands transition over smaller neighborhoods than coarse bands.

This is the pyramid blend of Chapter 6.6, Pyramids and wavelets with the hand-drawn mask replaced by an automatic quality weight.

fig-exposure-fusion-3
Figure 18.4.2. Exposure fusion drawn as a pyramid blend: the pyramid construction of Chapter 6.6, Pyramids and wavelets, with the mask replaced by an automatic quality weight. Each input exposure becomes a Laplacian pyramid and each per-pixel weight map a Gaussian pyramid; the two are multiplied and summed level by level into a fused pyramid that collapses to the final image. After Mertens et al. (exposure fusion).
Algorithm 18.4.1
# exposure fusion, the whole method
W = [contrast(I)**a * saturation(I)**b * wellexposed(I)**c for I in stack]
W = [w + epsilon for w in W]  # positive floor avoids a zero normalizer
W = normalize_across_stack(W)
Lp = [laplacian_pyramid(I) for I in stack]
Gp = [gaussian_pyramid(w) for w in W]
out = collapse([ sum(Gp[i][l] * Lp[i][l] for i in range(len(stack))) for l in levels ])

That is the entire algorithm, and it is about thirty lines in practice.

fig-exposure-fusion-weights
Figure 18.4.3. Interactive. One bracket, with each frame's three quality measures shown as images and their product as the weight map, then the naive per-pixel blend beside the pyramid blend. The naive result has visible seams and haloes at every edge where the winning frame changes; the pyramid result does not, and a level slider shows why, by blending only the coarse levels or only the fine ones. A comparison view puts the fusion beside a merge-and-tone-map result from the same bracket, which is usually close enough that the reader has to look for the difference. Interactive: adjust the three exponents, toggle the pyramid blend, set the number of levels, and compare against the radiance-map route.

18.4.3 Why it survives⧉

Exposure fusion is a useful baseline when the goal is a rendered photograph. It should be compared with a calibrated merge under the intended appearance and output conditions rather than assumed equivalent.

What it gives up is worth being clear about.

There is no radiance map, so the output is a picture and not a measurement. Everything the calibrated route buys outside photography, image-based lighting, photometry, physical simulation, is unavailable, which is the subject of Chapter 17.2, Camera response and radiance calibration.

The rendering depends on the bracket, weights and pyramid construction. The output can be edited afterward, but that does not recover the radiance or exposure information discarded by fusion. The source bracket is worth keeping for later reinterpretation.

The weights encode a preference, not a physical confidence estimate. The result can still be wrong: ghosts, halos, unstable colors and missing texture are measurable failures.

Motion still ghosts. Nothing about fusion helps with a moving subject, so the rejection machinery of Chapter 17.4, Ghosts and motion is needed just the same.

Fusion needs the same attention to registration, validity and changing content as other multi-frame methods.