BookNotesSlidesP-sets Syllabusdraft · v0.2.49.1
◀▲⌂▶
💬Comments welcome. To leave a note, select any text and click the note / highlight button that pops up — or open the panel with the tab at the top-right (‹). Notes are visible only inside our private review group.

17.5 Single-shot range and inverse tone mapping⧉

A single integration avoids disagreement between separately captured exposures. It still integrates motion and noise, and its readout can introduce rolling-shutter effects. Sensor HDR and inverse tone mapping address different limitations of a single delivered image.

Sensor HDR records a wider intensity range. Inverse tone mapping expands a rendered image for HDR presentation, sometimes estimating content lost to clipping or compression. It can preserve recorded information as well as introduce assumptions; the balance depends on the input and method.

17.5.1 Range without a bracket, in the sensor⧉

The section on HDR sensors compares time, gain and spatial multiplexing. For any delivered HDR frame, ask whether its channels share an integration window and which samples support each output value.

Lateral-overflow integration capacitors (LOFIC) store additional charge, extending highlight capacity (Akahane et al. 2006). Simultaneous gain paths can lower the effective shadow noise floor while preserving highlight headroom. Check the actual design: selectable conversion gain alone is not simultaneous multi-gain HDR.

Staggered exposures use different integration intervals. Spatially multiplexed subpixels may also integrate for different durations even when read as one frame. Both can show motion and flicker differences. An LED driven by pulse-width modulation at a few hundred hertz can be dark for all of a 1/2000 s exposure, so a short frame may show a lit traffic signal as off; the split-pixel design in Doing it in the sensor instead addresses this with a long integration that does not saturate. A long integration can average pulsed illumination more reliably, but no generic sensor label guarantees flicker-free recording.

Compare sensor and burst designs at the same capture time, photon budget and output requirement. Repeated short frames incur repeated read noise and registration costs. Hardware and software methods can be combined; there is no universal separation into sensor HDR for video and burst HDR for stills.

In CompPhotoRoom

The Merge module of Chapter 8.13, A raw developer in the browser runs this chapter's inverse-variance weighting on real brackets and shows each frame's share of the answer as an image, white where that frame is believed.

17.5.2 Inverse tone mapping: inventing the range⧉

In a rendered photograph, a clipped white window supplies a lower bound on exposure but no ordering among values above the threshold. Its boundary and surroundings still provide context. Unclipped pixels may retain useful relative intensity after response correction, while local tone mapping or quantization can make inversion uncertain.

Two broad approaches separate expansion of recorded tones from prediction of missing content.

Expansion operators map recorded tones into a larger display range and may expand highlights using spatial masks. If the original rendering curve is known and invertible over a region, its inverse can recover that region up to calibration and quantization. Clipping has no such inverse.

Learned reconstruction uses image statistics to estimate missing highlight intensities or structure. Eilertsen and colleagues train an HDR reconstruction network for this purpose (Eilertsen et al., 2017). Evaluate unclipped reconstruction and saturated-region inference separately; a plausible light source can still have the wrong radiance.

A generative prior can produce several plausible completions (Chapter 14.7, Generative super-resolution). Sampling variability reveals some ambiguity, but a model can also be consistently wrong across samples.

A useful observation constraint for a normalized linear capture is

$$z=\min(kL,z_{\rm sat}),\qquad z=z_{\rm sat}\Rightarrow L\ge z_{\rm sat}/k.$$

This noiseless model preserves a lower bound at saturation, not an exact value. With noise and a camera response, use the corresponding censored observation model. Expanded HDR can be useful for approximate image-based lighting or artistic rendering, but inferred intensities should not be presented as calibrated measurements. A rendered file format does not certify their accuracy.

[figure fig-inverse-tone-mapping not built]
Figure 17.5.1. Interactive. One clipped photograph, expanded three ways: an inverse curve with smooth extrapolation, a learned reconstruction, and a generative one drawn several times so the invented highlight structure changes between draws. Beside them, the real high dynamic range capture of the same scene, revealed on demand. A control varies how much was clipped, from a few percent to a blown sky, and the panel reports how much of the output's energy above the clipping point came from the model rather than from the file. Interactive: set the clipping level, switch methods, redraw the generative sample, and reveal the true capture.

17.5.3 Choosing among the three routes⧉

The part now has three ways to get range, and they are not interchangeable.

A bracket provides additional measurements at the cost of time and possible inconsistency. Use it when those measurements improve the intended reconstruction.

A sensor HDR mode can improve range within one delivered frame. Check its integration windows, noise, spatial sampling and clipping behavior.

Inverse tone mapping is useful for HDR presentation of an existing image. Keep the original, identify inferred regions, and distinguish appearance evaluation from radiometric validation.