BookNotesSlidesP-sets Syllabusdraft · v0.2.49.1
◀▲⌂▶
💬Comments welcome. To leave a note, select any text and click the note / highlight button that pops up — or open the panel with the tab at the top-right (‹). Notes are visible only inside our private review group.

18.0 TONE MAPPING AND DISPLAY⧉

Once we have captured a high-dynamic-range image (often called a radiance map), either using a high-quality sensor or multiple exposures, its range usually exceeds what can be reproduced on a display or print. A scene can span fifteen stops; a good screen or print shows six or seven in good conditions and can be as low as four when stray light lifts the blacks.

Tone mapping maps values from scene radiance to the final photograph. Imaging pipelines name the two ends. A scene-referred image holds values proportional to the scene's radiance, as a merged bracket does (Chapter 17.7, Encoding HDR images). A display-referred image holds the values a particular display should emit, or a print should reflect. Tone mapping is the step between the two. It is often done without reference to any particular display, aiming at a generic screen or print, although a display is always there implicitly, in the range and contrast the result is designed for. More and more, the display is taken into account explicitly: its peak brightness, its black level and the light of the room it sits in change what the rendering should be (Chapter 5.6, Preparing photographs for viewing, Chapter 18.10, HDR displays and display mapping). Figure 18.0.1 follows one image from the scene-referred state to four displays.

fig-image-state-pipeline
Figure 18.0.1. Image state through a modern pipeline. A scene-referred radiance map in linear light receives the creative grade while it is still scene-referred (a Look Modification Transform, in the terms of the film industry's ACES framework): that is a choice. Then one Output Transform per display, the rendering for that display, each following from the display's peak and its viewing surround: a cinema projector at 48 cd/m² in a dark room (system gamma 1.5), an HDR television at 1,000 cd/m², a standard monitor at 100 cd/m² (gamma 1.2), and a phone outdoors, with a lower gamma and lifted shadows against its bright surround. The display-referred thumbnails are shown relative to each display's peak. Radiance map by Paul Debevec; the transforms are our simple stand-ins.

The previous part (Part 17, High dynamic range capture) ends with a measurement and an account of where it is reliable; this part spends it.

Whereas HDR capture is usually an objective measurement proportional to physical units, tone mapping is often a subjective endeavor. In some circumstances, the goals can be tied to psychophysics, especially when realistic simulations are sought. For example, a realistic driving simulation seeks to reproduce visibility properly. But in most photographic situations, the goal is subjective: we want images to look good. Yet, in both cases, knowledge of the human visual system helps set the principles.

18.0.1 Three things called tone mapping⧉

Three different problems share the name, with different criteria of success, and much of the confusion in the field comes from treating them as one.

The first is range compression. Fourteen stops have to become six, or ten. This is a constraint rather than a goal: it says that something must be discarded, but not what the result should look like, and its criterion of success is negative, that nothing important clipped and the image is viewable. Most of the tone-mapping research of 1995 to 2010 posed the problem this way, which is part of why it never converged: a constraint has no optimum. A sharpened version of the goal changes the answer. Medical imaging, remote sensing, surveillance and inspection want every detail visible at once, and for them strong local operators are correct, halos included, since a halo is not a failure when the task is detection. That goal justifies the most aggressive local methods, and it is rarely the photographer's.

The second is appearance reproduction: the viewer of the picture should have the experience an observer standing in the scene would have had. This is the problem Hunt set out in The Reproduction of Color Hunt, The Reproduction of Color, and it is the only one of the three with an answer that can be computed in principle, since appearance can be measured by matching experiments. Its central and non-obvious point is that reproducing the scene's luminance ratios exactly gives the wrong appearance. The viewing conditions differ from the scene's in specific ways, each with a measurable effect, and the reproduction must depart from the measurement in order to match in appearance. Chapter 18.1, Tone mapping and perception lists those ways.

The third is the look: what people prefer rather than what was there. Memory colors, the shoulder inherited from film, a manufacturer's rendering, a creative grade. There is no ground truth, success is preference, and preference varies with culture, content and decade. The look is a legitimate goal; the mistake is only in confusing it with the second. A faithful rendering of a backlit portrait is a dim face against a bright window, while the preferred one brightens the face and lets the window run a little brighter than it should, and in informal comparisons viewers tend to choose the second. Camera pipelines and editors are mostly tuned for preference; faithful reproduction matters where matching the original is the point, in film restoration, scientific imaging and some medical uses.

Figure 18.0.2 renders one radiance map for each goal.

fig-three-goals
Figure 18.0.2. One radiance map, a sunset behind vines and rooftops, rendered for each of the three goals. Maximum information: a strong local operator, Durand and Dorsey's two-scale method with the base squeezed to 3:1 and the detail boosted, so every region can be read and halos are accepted. Appearance: a global rendering as the scene would have looked at dusk, with a system gamma of 1.2 and a gentle shoulder, the sky bright and the ground dark. Preferred look: the same with lifted shadows, a deeper blue sky and warmer midtones. Each succeeds by its own criterion; none is the best. Radiance map by Paul Debevec, our renderings.

18.0.2 How this part is organized⧉

Chapter 18.1, Tone mapping and perception sets out what the visual system does with a picture: the four reasons a faithful reproduction must depart from the scene, and the spatial vision that local operators exploit. Chapter 18.2, Global tone mapping covers single curves, from Ward's scale factor and Reinhard's photographic operator to histogram adjustment, and why a single curve cannot win. The local methods follow, one mechanism per chapter: Chapter 18.3, Two-scale and bilateral tone mapping, Chapter 18.4, Exposure fusion, which skips the radiance map altogether, Chapter 18.5, Local Laplacian filtering and other multiscale methods, and Chapter 18.6, Gradient-domain tone mapping. Chapter 18.7, Tone mapping as perceptual optimization derives the curve from a model of vision and of the display instead of choosing it, and Chapter 18.8, Display rendering transforms and color turns to the fixed rendering transforms of modern pipelines, where the behavior of hue and saturation through the compression turned out to be the hard part. Chapter 18.9, High dynamic range in traditional photography tells the same story in the darkroom, where most of these ideas started. Chapter 18.10, HDR displays and display mapping follows the picture to displays whose range differs from one viewer to the next, Chapter 18.11, Evaluating tone mapping asks how any of it can be judged, and Chapter 18.12, HDR TLDR summarizes both HDR parts.

Contents of this part

▸ full collapsible outline of this part