35.1 Flash photography⧉
A flash is the simplest piece of computational illumination there is: a single, brief, known burst of light fired in sync with the shutter. For a century it was used in the obvious way, to add light when the scene was too dark, and it earned a deserved reputation for ugliness. An on-camera flash points straight down the lens axis, so it flattens every shape it touches, blows out near objects, leaves far ones black, throws a hard shadow onto the wall behind, and turns the back of the eye red. The computational-photography move is to stop thinking of the flash as the light and start thinking of it as a second condition. You now have two images of the same scene under different, controlled illuminations: a warm, atmospheric, but dark and noisy ambient frame, and a bright, sharp, but harsh flash frame. The interesting question is no longer "which one is better" but "how do I fuse them so the result keeps the ambient mood and the flash's clean detail at once." That reframing, plus the physics of where flash light goes (down the axis, falling off as $1/d^2$), organizes the chapter.
35.1.1 Flash / no flash⧉
Take two photographs of the same scene back to back: one with the flash off, one with it on. The no-flash image records the ambient lighting, such as the warm tungsten glow of a restaurant or the blue cast of dusk, but because the ambient light is dim, the camera had to choose between a long exposure (motion blur) and a shorter exposure with higher gain (fewer photons and more visible noise), so the no-flash frame is typically dark, noisy, and soft. A flash can provide a sharper, higher-SNR guide, because the flash dumped a lot of clean light in a millisecond, but it has lost the ambient mood: the warm restaurant now looks flat and harshly lit, colors are wrong, and a harsh shadow has appeared. Neither image is the one you want. The one you want has the no-flash ambience and color and the flash detail and low noise.
Petschnigg et al. (2004) and, in parallel, Eisemann and Durand (2004) showed how to get it, and the key tool is the cross-bilateral (or joint bilateral) filter from Chapter 10.1, Bilateral filtering. A bilateral filter smooths an image while preserving its edges by weighting neighbors by both spatial and range (color) distance; the cross-bilateral variant computes those range weights from a second, guide image rather than from the image being filtered. This works for a simple reason: you want to denoise the noisy no-flash image without blurring its edges, but the no-flash image is too noisy to find its own edges reliably. The flash image, however, has the same scene's edges rendered cleanly. So you filter the no-flash color using the flash image as the edge guide, with a joint bilateral filter, and the flash's sharp edges stop the smoothing from crossing real boundaries while the ambient color and tone are preserved. Detail that survives only in the flash frame can then be transferred onto the denoised ambient image as a detail layer (a ratio of the flash image to its own bilateral-smoothed version), so fine texture comes from the flash while large-scale color and lighting come from the ambient (Figure 35.1.2, referenced).
Eisemann and Durand's framing makes the decoupling explicit: they separate each image into a large-scale layer (overall color and lighting "mood") and a detail layer, take the mood from the no-flash image and the detail from the flash image, and recombine them. The result is an intrinsic relighting of the scene that keeps the ambient atmosphere while borrowing the flash's clarity. Both papers also handle the flash's two signature artifacts: the harsh cast shadow (detected from a weak added-flash response after exposure normalization, then handled using ambient information) and specular highlights the flash strikes off glossy surfaces.
Figure 35.1.1 runs Eisemann and Durand's pipeline in the page on their own photographs.
The last stage of Figure 35.1.1 runs the shadow treatment step by step.
The modern learned successors to these hand-built fusion rules need data, specifically many real flash/ambient pairs, and that is what Aksoy et al. crowdsourced in their FAID dataset Aksoy et al. 2018 (FAID flash/ambient dataset): thousands of flash / no-flash image pairs captured in the wild by hundreds of people, now a standard training and benchmark set for learned flash photography (denoising, illumination decomposition, relighting). It is cataloged with the book's other corpora in Chapter 45.16, Datasets.
35.1.2 Ramesh's multiflash⧉
Raskar, Tan, Feris, Yu and Turk (2004) turned the flash's worst habit, casting hard shadows, into a feature, and used it to find a scene's geometry rather than its color. Their multi-flash camera mounts four small flashes around the lens, at left, right, top, and bottom, and fires them one at a time, capturing one image per flash. Because each flash sits a few centimeters off the lens axis, it casts a thin shadow just past every depth discontinuity, the silhouette where a near object occludes a farther surface. The geometric key: the shadow always falls on the side of the silhouette opposite the flash. The left flash throws a shadow sliver to the right of an object's edge; the top flash throws one below; and so on (Figure 35.1.5).
That gives a simple detector for depth edges. At a true depth discontinuity, some flash will cast a shadow there, whichever one is on the occluding side, so a pixel that goes dark in at least one of the four flash images, in the characteristic shadow-opposite-the-flash pattern, sits on a silhouette. Texture and material edges often retain similar structure under these nearby flash directions (they are not occlusions, they cast no parallax shadow), so the method rejects them. Combining the four images, for example by taking the ratio of each flash image to the per-pixel maximum and then tracing where the shadows begin, yields an estimate of depth edges, with failures from specularities, weak signals, overlapping shadows, and motion, the occluding contours of the scene, with the intensity and texture edges filtered out. They called the device a non-photorealistic (NPR) camera, because once you have the scene's silhouettes you can render it in stylized, line-drawing-like ways: outline the objects, abstract away texture, or composite the depth edges back over a de-emphasized interior. By offsetting a known light and reading the shadows, a flat camera recovers a piece of 3-D structure that no single image contains.
35.1.3 Removing flash artifacts⧉
Even when you only want the flash image, it arrives with artifacts that the pair of images lets you remove. Agrawal, Raskar, Nayar and Li (2005) gave a general gradient-domain treatment. The observation is that a flash and an ambient image of the same scene share the same true scene structure but differ in the artifacts each introduces: the flash adds specular highlights and hot spots off glossy surfaces and reflections in windows, while the ambient adds noise; and an artifact gradient in one image is typically not aligned with the corresponding scene gradient in the other. They use gradient coherence to identify artifacts and project gradients from the contaminated image onto the direction supplied by the cleaner guide. For self-reflection removal, the ambient gradients are projected onto the flash gradients; flash-highlight removal uses the ambient structure to constrain the repair. They then reintegrate (a Poisson solve, as in Chapter 10.8, Poisson image editing) to reduce the targeted artifacts. The same machinery removes reflections off glass (shoot through a window with and without flash; the flash reflection is a gradient present in one image but not the scene) and the flash's own hot spots.
This is a recurring pattern: two aligned images under different illumination can distinguish some lighting-dependent artifacts from shared scene structure. Genuine shading changes, motion, and misregistration also create inconsistent gradients, so inconsistency alone is not proof of an artifact. Noise is inconsistent (random), specular highlights are inconsistent (they move with the light), reflections are inconsistent (they depend on the illumination geometry), and the scene, the thing you want, is the consistent part. Flash/no-flash denoising, multi-flash depth edges, and flash de-reflection are three faces of that single idea.
35.1.4 Dark flash and covert infrared photography⧉
The flash/no-flash story has one stubborn flaw: the flash is blinding and disruptive. It dazzles the subject, ruins the candid moment, and is forbidden in museums, theatres, and wildlife photography. Krishnan and Fergus (2009) proposed a simple fix: move most of the flash energy outside the visible band. A silicon camera sensor is sensitive well beyond the visible spectrum, into the near-ultraviolet and the near-infrared, bands the human eye cannot see at all. So build a dark flash that emits in those out-of-band wavelengths (just below 400 nm and just above 700 nm): it floods the scene with light the camera records brightly with much less visible brightness, reducing dazzle and it remains usable in a "dark" room (Figure 35.1.6).
The dark-flash image is, like an ordinary flash image, sharp and low-noise, but it is recorded in the wrong wavelengths, so its colors are meaningless (skin, foliage, and fabrics reflect UV and IR quite differently from how they reflect visible light). It plays exactly the role the flash image played in flash/no-flash fusion: a clean detail and edge guide for denoising the noisy, dim, but correctly-colored visible ambient image. Krishnan and Fergus fuse the two with a cross-channel scheme. They borrow high-frequency detail and edge structure from the dark-flash image, keep low-frequency color from the visible ambient image, and exploit statistical relationships between the bands to suppress the places where UV/IR reflectance diverges from visible. The result is a flash-quality photograph taken with a much less visible flash. Spectral mismatch remains a reconstruction limitation, and low visibility does not by itself make a source suitable for every subject or venue.
Photographing people who cannot tell they are being photographed, by lighting them in infrared, predates the consumer dark flash and was used for surveillance. Infrared film and IR illuminators were used for covert night surveillance through much of the twentieth century. The East German Stasi's clandestine night photography is the notorious example, and military "active IR" night-vision systems are the same idea industrialized. The optics are identical to the dark flash; only the intent differs. It is a useful reminder that computational illumination, like any imaging capability, is dual-use: the same physics that lets a wildlife photographer avoid spooking an owl lets a watcher photograph a person in the dark. The book returns to the surveillance and privacy stakes of active imaging in the dual-photography chapter (Chapter 35.7, Dual photography).
