38.0 REVEALING THE INVISIBLE🔗⧉

The pictures in this part are of things you cannot directly see. The world is full of faint, unintended traces, such as light that bounced one extra time, motion too small for the eye, or a scene hidden around a corner. With the right capture and computation, each can become an image. Ordinary scenes turn out to contain accidental cameras: a window can act as an aperture, a houseplant can cast informative shadows, and a person's eye can act as a curved mirror, each encoding part of its surroundings (Chapter 38.1, Accidental cameras, Chapter 38.2, Reflections in the eye). Motions below the threshold of vision, such as a pulse in a face or a structure flexing, are magnified until they are obvious (Chapter 38.3, Motion and video magnification), and the sound that faintly vibrates a chip bag is read back off silent video as a visual microphone (Chapter 38.4, Visual microphone). And light that has scattered diffusely off walls still carries information about what lies out of the line of sight, which corner cameras and active and passive non-line-of-sight methods reconstruct (Chapter 38.5, Corner camera, Chapter 38.6, Active non-line-of-sight, Chapter 38.7, Passive non-line-of-sight), a story that extends past visible light entirely, into mm-wave and Wi-Fi sensing (Chapter 38.8, Mm-wave, wifi).

The unifying idea is that measurement plus a model of how the trace was formed can recover a signal no single pixel was ever pointed at. Each chapter pairs an unlikely sensor, such as a reflective surface, a brightness time series, or a scattering wall, with the forward model that explains what it encodes, then inverts it. It is computational photography at its most forensic: extracting the picture hidden inside the one you took.

Contents of this part

• 38.1 Accidental cameras
  • 38.1.1 The accidental pinhole: a window is a camera
  • 38.1.2 The accidental pinspeck: the anti-pinhole
  • 38.1.3 The occluder as a crude lens, and recovery as deconvolution
  • 38.1.4 Corners and doorways: an edge that resolves the hidden room
  • 38.1.5 Where else the world hides a camera
• 38.2 Reflections in the eye
  • 38.2.1 The cornea as a catadioptric mirror
  • 38.2.2 The geometry: from a corneal pixel to a direction in the world
  • 38.2.3 What the recovered reflection is good for
  • 38.2.4 Eyes for relighting
• 38.3 Motion and video magnification
  • 38.3.1 The Lagrangian precursor: track, then exaggerate
  • 38.3.2 Eulerian video magnification: amplify the time series at each pixel
  • 38.3.3 Why amplifying brightness amplifies motion
  • 38.3.4 Phase-based magnification: move the motion into phase
  • 38.3.5 What it reveals: vital signs, structures, materials, modes
• 38.4 Visual microphone
  • 38.4.1 From sub-pixel motion to a sound waveform
  • 38.4.2 Bandwidth: high-speed cameras and the rolling-shutter trick
  • 38.4.3 How good is the copy? The object's frequency response
  • 38.4.4 The active cousins: laser vibrometry and interferometry
• 38.5 Corner camera
  • 38.5.1 The edge as a one-dimensional aperture
  • 38.5.2 From a faint gradient to a usable signal
  • 38.5.3 What the corner can and cannot tell you
• 38.6 Active non-line-of-sight
  • 38.6.1 Third-bounce geometry and time-of-flight
  • 38.6.2 The hardware: photographing light in flight
  • 38.6.3 From back-projection to fast, exact inversion
  • 38.6.4 The trade, stated plainly
• 38.7 Passive non-line-of-sight
  • 38.7.1 The occluder is what makes it solvable
  • 38.7.2 A deconvolution where the lens is unknown
  • 38.7.3 One dimension: the corner camera
  • 38.7.4 Two dimensions from a single photo: computational periscopy
  • 38.7.5 Active versus passive, the ledger
• 38.8 Mm-wave, wifi
  • 38.8.1 Why radio walks through walls
  • 38.8.2 Time of flight, again: radar is NLOS with a longer wave
  • 38.8.3 From a radio smear to a human skeleton: the learned map
  • 38.8.4 What it sees, and what it costs us

▸ full collapsible outline of this part