17.1 Dynamic range and multiple-exposure capture🔗⧉

Dynamic range is the ratio between the largest and smallest signal a sensor records. At the bright end, charge capacity or the readout chain limits the recorded signal. At the dark end, noise sets a gradual loss of precision rather than a hard black threshold. The usable dynamic range depends on the minimum signal-to-noise ratio (SNR) accepted for the task (Chapter 2.8, Noise, signal-to-noise ratio and dynamic range). That chapter develops both ends: full-well capacity at the top (the maximum number of electrons a pixel can hold), read noise and photon noise at the bottom. The engineering definition places the floor where the signal equals the noise (0 dB), but that is too low for photographic purposes, where 30 dB is considered close to perfect and even 15 dB is severely noisy. Figure 17.1.1⛶ fullscreen shows examples. Like depth of field, quantitative dynamic range is a question of tolerance, to noise in this case. Recall that bright scene values that saturate the photoreceptor may still be read below the maximum because of electronic noise. The ability to capture high dynamic range can further be limited by the veiling glare inside the lens, which lifts the darkest tones before the sensor sees them.

When the dynamic range of a scene exceeds that of the sensor, we capture multiple exposures to span the range. We may need to register these images in space (similar to denoising). Chapter 13.2, Denoising by averaging aligns and averages a burst of equal exposures, Part 12, Matching pixels across space and time covers the registration itself, and Chapter 17.4, Ghosts and motion deals with what moves between the frames of a bracket. We also need to calibrate the pixel values photometrically to bring them into the same absolute linear range proportional to scene-radiance values $L(x,y)$, which may require the calibration of the film or digital camera response curve (the beauty curve in particular, Why scene-linear looks dull in Part 8, Basic image processing and ISP; Chapter 17.2, Camera response and radiance calibration recovers it). Motion, clipping in every frame, or inadequate shadow exposure can leave parts of that estimate uncertain.

17.1.1 The gap between the scene, the sensor and the screen🔗⧉

Distinguish the range within one scene from the much larger range encountered across daylight, interiors and night. Likewise, visual adaptation across time is different from simultaneous contrast sensitivity. A sensor's capture range and a display's visible contrast also depend on noise criteria, viewing conditions and hardware; one fixed ratio does not describe all of them.

With that caveat, some orders of magnitude help. A view from a dim interior through a window onto a sunlit street can span around 100,000:1, about 17 stops, between the brightest diffuse surface and the darkest shadow that still shows detail, before counting the sun or specular highlights.

Following the light from its source shows where such a number comes from. The disk of the sun has a luminance of about $1.6 \times 10^{9}$ cd/m², far beyond anything else in a photograph. A clear blue sky is around 5,000 to 10,000 cd/m², some two hundred thousand times dimmer. A white wall in direct sun receives about 100,000 lux and, reflecting most of it, has a luminance around 25,000 cd/m². In the shade, a surface sees the whole sky but not the sun: a uniform sky of luminance $L$ over a full hemisphere delivers an illuminance of $\pi L$, about 15,000 lux, so a mid-gray surface there is around 2,500 cd/m², ten times dimmer than the sunlit wall. Indoors, a surface near a window sees only the part of the sky the window frames. A one-square-meter window three meters away subtends about 0.1 steradian, so the illuminance drops to a few hundred lux, some forty times less than in the open shade, and a mid-gray surface is around 60 cd/m². Many points in the room do not see the window at all and are lit only by light bounced off the walls and ceiling, a few lux to a few tens of lux; a dark fabric there, reflecting five percent, is around 0.2 cd/m². From the sunlit wall outside to that fabric is about 25,000 / 0.2, or 125,000:1: the 100,000:1 above, reached by a chain of ordinary factors, each of ten or forty, and without the sun or the sky ever appearing in the frame.

No single exposure of such a scene holds both ends: Figure 2.8.14⛶ fullscreen develops one scene at three exposures.

The eye covers a far wider range, but not at once (Chapter 3.3, Adaptation and constancy). Across its states of adaptation, vision works from about a millionth of a candela per square meter, the threshold of fully dark-adapted vision, to sunlit snow at $10^{4}$ to $10^{5}$ cd/m², some ten or eleven orders of magnitude, well over thirty stops. At any one operating point it distinguishes far less, roughly four orders of magnitude, and it sees a high-contrast scene well because it moves: each fixation adapts to the part of the scene it lands on. A camera has to record the whole scene in one exposure, which is why 100,000:1 is a problem for it and not for the eye.

A current large-sensor camera at base ISO (the lowest ISO, i.e. the lowest gain) records roughly 12 to 14 stops between clipping and a noise floor at a signal-to-noise ratio of one, and a phone sensor fewer. That is the engineering definition, with the floor at 0 dB, where the signal equals the noise, which nobody would call a usable shadow. Raising the floor costs range quickly. With photon noise and read noise, the dimmest signal reaching an SNR of $q$ is $N_{\min}=(q^2+\sqrt{q^4+4q^2\sigma_r^2})/2$ electrons (Chapter 2.8, Noise, signal-to-noise ratio and dynamic range). For a full-frame pixel with a full well of 50,000 electrons and 3 electrons of read noise, the range is 13.8 stops at 0 dB, 11.6 stops at 10 dB (an SNR of about 3) and 8.9 stops at 20 dB (an SNR of 10). For a phone pixel with 6,000 electrons and 1.5 electrons of read noise, the same three floors give 11.5, 9.0 and 5.9 stops. At 20 dB the floor is set almost entirely by photon noise, about 100 electrons whatever the read noise, so the range is little more than the full well divided by 100, and what buys clean shadows is a deeper well, which means a larger pixel. Figure 17.1.2⛶ fullscreen shows how it falls with ISO for ten cameras.

The most widely used sensor measurements are DxOMark's (DxOMark camera sensor database). They report dynamic range in stops (EV), from saturation down to the level where the signal equals the noise, as a function of ISO, in two modes. Screen mode measures each pixel as it is. Print mode normalizes every camera to the same output size, 8 megapixels, as if the image were downsampled for a print of fixed size; averaging pixels lowers the noise, so a sensor with more pixels gains about half a stop in Print mode for every doubling of its pixel count, at DxOMark's floor where signal equals noise; at a floor high enough that photon noise dominates, the gain approaches a full stop per doubling. Screen mode compares pixels, Print mode compares pictures, and the two can rank cameras differently. The best full-frame sensors reach close to 15 stops in Print mode at base ISO, and above the lowest ISOs a sensor loses close to a stop of range for each doubling of ISO. Bill Claff's Photons to Photos charts use a stricter floor, set by what is visible in a print of a given size, and so report smaller numbers for the same cameras.

Figure 17.1.3⛶ fullscreen computes such curves from the noise model alone, for four typical sensors, and leaves the floor and the normalization to the reader.

The ranges in this chapter move between stops, decibels and plain ratios. The converter of Chapter 2.8, Noise, signal-to-noise ratio and dynamic range is reproduced here for going from one to another (Figure 2.8.5⛶ fullscreen).

A standard-range display in good conditions (dim room, no glare) shows perhaps 1,000:1, about 10 stops, and much less in daylight, when reflected room light raises its black. Video projectors in a classroom can be as low as 10:1. A glossy print reflects roughly 100:1 to 250:1 if you avoid the reflection angle. Each number depends on the criterion used to define its dark end, which is why they should be compared with care.

Use a simplified linear observation model after black-level subtraction. The exposure factor $k_i$ includes shutter time, aperture transmission and calibrated gain; it converts scene radiance to the chosen signal units:

Each frame scales the same radiance, adds noise and clips at its recording limits. This approximation omits spatially varying optics, motion during exposure and nonlinear processing. Gain changes the signal units and noise model; it does not collect more photons.

17.1.2 Bracketing, and which parameter to vary🔗⧉

The principle of multiple-exposure high-dynamic-range photography is simple. Choose exposure factors with overlapping usable ranges, take photos, then merge into a floating-point radiance estimate. Its scale is usually relative unless an absolute radiometric calibration is supplied. Formats such as OpenEXR can preserve scene-linear values for later rendering (Chapter 8.11, File formats and compression).

On a logarithmic radiance axis, changing exposure slides each frame's useful interval. One stop moves it by a factor of two. Overlap matters because neighboring exposures need common usable signal for calibration and a smooth change in which measurement is most precise. A gap leaves intensities that are clipped in one frame and poorly measured in the next. The merged floating-point values can exceed one; that is normal for a scene-linear representation whose scale is relative.

The exposure factor $k_i$ can change in several ways, but each control changes something else as well. A useful capture sequence considers those side effects before choosing how widely to space its exposures.

For a conventional bracket, keep aperture, focus and gain fixed and vary shutter duration. This preserves the lens's focus blur and usually provides predictable exposure ratios. It also changes motion blur and capture timing, so the frames are not identical apart from brightness.

Under steady illumination and away from clipping, expected photon count is proportional to exposure duration. A longer exposure improves photon statistics but integrates more subject or camera motion and accumulates more dark current. Flickering light and rolling-shutter timing can violate a simple global exposure ratio.

Changing aperture changes depth of field, diffraction and often vignetting. Changing ISO changes gain and possibly input-referred read noise or saturation headroom, without changing the photons captured at fixed shutter and aperture. A neutral-density filter attenuates light, including flash, but its transmission and color should be calibrated. These controls can be useful in a designed capture sequence; they are not interchangeable with shutter duration.

Set the shortest exposure to preserve important highlights and the longest to reach acceptable shadow SNR without excessive motion blur. Choose intermediate exposures with enough overlap for calibration and merging. Stop adding frames once they provide little useful coverage or noise reduction for their time cost.

A tripod reduces camera motion but does not freeze subjects, foliage or illumination. Handheld brackets need registration; even tripod brackets need consistency checks. Ward's median-threshold bitmap (MTB) aligns differently exposed images using thresholded intensities and excludes values near the median (Ward 2003 (MTB)); Chapter 17.4, Ghosts and motion shows it at work. Clipping, noise and changing content can still corrupt the comparison.

Stops make the sequence easy to reason about: $1/1000$, $1/250$ and $1/60$ second form an approximately two-stop progression. The same exposure increase could be obtained by opening the aperture two stops, but then depth of field and the blur operator change. Flash gives another reason to distinguish controls. Within ordinary flash-sync operation, changing shutter duration mainly changes the continuous-light contribution; a neutral-density filter attenuates both ambient and flash light. The desired bracket must therefore account for which source the control actually changes.

17.1.3 Doing it in the sensor instead🔗⧉

Sensors can combine measurements with different integration intervals, conversion gains or sensitivities into one delivered HDR frame. A single output frame does not necessarily mean all its measurements cover the same time interval. Check the sensor's acquisition and readout design (Part 34, Computational sensors).

Time multiplexing combines shorter and longer integrations. Digital-overlap HDR interleaves their readout; it should not be described as simply reading one unchanged exposure twice. Different integration windows can produce motion or illumination inconsistencies even when packaged as one output frame.

Gain multiplexing can read charge collected during a common integration through paths with different conversion gains. One path favors low input-referred read noise and another highlight capacity. This avoids the inter-exposure mismatch of a bracket when the integrations are shared. Ordinary selectable dual-conversion-gain operation does not by itself imply that both gains are delivered simultaneously.

Spatial multiplexing assigns different sensitivities or collection areas to neighboring sites or subpixels. Nayar and Mitsunaga's spatially varying exposure array reconstructs a wide-range image from such a pattern (Nayar & Mitsunaga 2000). The sampling and noise tradeoffs depend on the layout. Overflow-capacitor designs instead add charge storage; logarithmic or companding responses allocate output precision nonuniformly across intensity. None removes the need for calibration or a noise model.

Split-pixel sensors, common in automotive cameras, are a concrete spatial design. Each pixel holds a large photodiode and a small one. For the same light the small one collects far fewer electrons, so it saturates much later and records the highlights while the large one records the shadows, and the two are merged into one value per pixel. The motivating scene is a car leaving a tunnel: dark walls and daylight at the exit in one frame, with no time to bracket, plus LED signs and headlights driven by pulses that a short exposure can miss entirely. Because the small diode can integrate over a long window without saturating, it can record those pulses where a short exposure of the large diode would not. The two diodes differ in noise, response and color, so the merge needs its own calibration.

Compare integration timing, spatial sampling, read noise and saturation capacity separately. Shared integration avoids one source of ghosts, but does not eliminate motion blur or rolling-shutter distortion. A software burst adds repeated readout, computation and alignment costs.

17.1.4 Veiling glare, the limit bracketing cannot remove🔗⧉

Bracketing extends the range the sensor can record at each pixel. It does nothing for light that arrives at the wrong pixel. Inside the lens, a small fraction of the light from every bright point is reflected between glass surfaces or scattered by the glass, the coatings, the barrel and the sensor, and spread over the image (Chapter 22.10, Glare suppression). This veiling glare is part of every exposure, in proportion to the exposure, so the merge treats it like any other light and it stays in the radiance map. Near a window or the sun, the glare from the bright region can exceed the true radiance of a dark region beside it, and the merged value there measures the glare rather than the scene. McCann and Rizzi photographed test targets of known range and found that the darkest areas were recorded far brighter than they were, and that adding exposures did not change this: glare, not the sensor, set the range the camera could measure McCann & Rizzi 2007.

Like a blur, glare can be described by a point spread function, but one with a small, sharp core and a broad, faint tail (Figure 22.10.5⛶ fullscreen). A simple model writes the recorded radiance map $\hat L$ as

where $k_g$ is a broad kernel that sums to one and $\alpha$ the fraction of the light it carries, a few percent or less in a well-coated lens. The tail is far below the core everywhere, so it hardly matters where the scene is moderately bright. It adds up from every bright pixel, though, and dominates where a dark region is surrounded by bright ones. As an example, suppose a window 10,000 times brighter than the room fills a fifth of the frame, $\alpha=1\%$, and the tail spreads evenly over the frame. The glare at a point in the room is then $\alpha$ times the mean radiance of the frame, $0.01\times(0.2\times10{,}000+0.8)\approx20$ times the room's own radiance: the room is recorded about twenty times too bright, and the measured contrast between window and room drops from 10,000:1 to about 500:1.

The basic correction measures $k_g$ and $\alpha$ once, from a high-dynamic-range photograph of a small bright source in a dark surround, and subtracts the predicted glare from the merged map. Since the glare is a small fraction of the light, we can predict it from the recorded map itself:

Using $\hat L$ instead of $L$ inside the glare term is accurate to first order in $\alpha$. The correction is only as good as the model. Glare varies across the field and with the aperture; light from sources outside the frame produces glare but is not in the map; pixels clipped even in the shortest exposure make the bright sources look dimmer than they are, so their glare is underestimated; and ghosts, the discrete reflections of a bright source, are not a smooth kernel. Subtraction also removes the glare's mean but not its photon noise, so the corrected dark regions are noisier than a photograph without glare would be. The space-varying and deconvolution versions are in Chapter 22.10, Glare suppression.

Talvala, Adams, Horowitz and Levoy measured the glare instead of modeling it Talvala et al. 2007. They placed an occluding mask with a fine pattern between the camera and the scene and took a series of photographs with the mask shifted between shots. A pixel behind an opaque part of the mask receives no direct light from the scene, only glare, so those pixels sample the glare, and the samples from all the mask positions give the glare everywhere, ready to subtract. It costs many exposures and a mask in front of the scene, and the blocked bright regions also contribute less photon noise to the shadows. Chapter 22.10, Glare suppression shows the results and describes the other methods, including a mask inside the camera and learned flare removal.

17.1.5 Two caveats: single-shot range, and where this is used🔗⧉

First test a single raw file at the intended output size. Bracketing helps when it supplies cleaner shadows or unclipped highlights that the single capture lacks. Fill flash and graduated neutral-density filters can also reduce scene contrast at capture; dodging and burning change its rendering afterward.

These choices have photographic precedents. Le Gray combined negatives to retain sea and sky, while graduated neutral-density filters and fill flash change the contrast presented to the camera. Dodging, burning and the Zone System help control how recorded tones become a print; Chapter 18.9, High dynamic range in traditional photography, in the tone-mapping part, treats each of them.