2.0 FUNDAMENTALS of Imaging🔗⧉

A photograph records light that has traveled from a source, interacted with a scene, and reached a sensor or eye. Light leaves a source, such as the sun, a lamp, or a phone screen, and travels until it strikes objects in a scene, scatters according to what those objects are made of, and a sliver of it finally passes through a lens onto a sensor (or, in the eye, onto the retina). The sensor counts photons; the camera turns those counts into a grid of numbers; a visual system, silicon or biological, interprets the result (Figure 2.0.1⛶ fullscreen). Photography means, almost literally, writing with light, from the Greek phōs ("light") and graphē ("drawing").

This part follows that journey up to the sensor, because every later technique in the book depends on it. We begin with Chapter 2.1, Light and optics: what it is (wave, ray, or photon), how it interacts with matter, and how we measure it (the first chapter). Then we ask how a camera turns the light filling a room into a flat picture: Chapter 2.2, Pinhole image formation and linear perspective to Chapter 2.3, Lens image formation and the perspective it imposes. A pixel value is really an integral Chapter 2.4, Image measurements as integrals over the plenoptic function, and two of the most visible photographic effects are that integral at work: Chapter 2.5, Depth of field, a blur over the aperture, and Chapter 2.6, Motion blur, a blur over the exposure. From there we reach Chapter 2.7, Sensors: photosites, CCD vs CMOS and Chapter 2.8, Noise, signal-to-noise ratio and dynamic range, where this part ends. The abstract framing those threads lead to, imaging as a linear system and as a Bayesian inverse problem, is taken up later in the book, in Chapter 6.7, Image formation as a linear system and Chapter 7.1, Imaging as Bayesian inference with probabilistic priors. The eye and brain interpret those measurements through adaptation, trichromacy and visual processing (Part 3, Visual perception and color). Color measurement, encoding and reproduction are developed in Part 4, Color spaces and color technology. The craft of a real camera as an instrument follows in Part 5, Photography.

Contents of this part

• 2.1 Light and optics
  • 2.1.1 Light: rays, waves, and the spectrum
  • 2.1.2 How light is created
  • 2.1.3 Light power and brightness
  • 2.1.4 Summing coherent vs. incoherent light
  • 2.1.5 Reflection, refraction, and what happens at a surface
  • 2.1.6 Light-matter interaction is largely resonance
  • 2.1.7 Polarization
  • 2.1.8 The color of objects: illumination times reflectance
  • 2.1.9 Where color comes from
  • 2.1.10 The BRDF and the look of materials
  • 2.1.11 Radiometry: radiance, irradiance, exposure, and falloff
  • 2.1.12 Global illumination
  • 2.1.13 Wave effects, diffraction, and the diffraction limit
• 2.2 Pinhole image formation and linear perspective
  • 2.2.1 Pinhole imaging and the perspective projection
  • 2.2.2 Homogeneous coordinates
  • 2.2.3 Camera in a general configuration
  • 2.2.4 Intrinsics, extrinsics, and what cropping really does
  • 2.2.5 What perspective preserves, and what it destroys
  • 2.2.6 Wide-angle distortion: spheres bulge and faces stretch at the edges
  • 2.2.7 Photography with focal length: framing, magnification, and compression
  • 2.2.8 Depth, ray length, and unprojection
• 2.3 Lens image formation
  • 2.3.1 Single lenses: refraction and Snell's law
  • 2.3.2 Thin lens optics
  • 2.3.3 Aperture and the f-number
• 2.4 Image measurements as integrals over the plenoptic function
  • 2.4.1 The plenoptic function
  • 2.4.2 The pixel integral
  • 2.4.3 Exposure values (EV) and stops
  • 2.4.4 Splitting the integral
  • 2.4.5 Attempts to visualize the plenoptic function
• 2.5 Depth of field
  • 2.5.1 The circle of confusion
  • 2.5.2 Depth of field versus depth of focus
  • 2.5.3 Choosing the maximum circle of confusion
  • 2.5.4 The double cone, and the near and far limits
  • 2.5.5 Hyperfocal distance
  • 2.5.6 The surprising invariance
  • 2.5.7 Background blur beyond the focus plane
  • 2.5.8 Defocus is not a blur of the image
  • 2.5.9 Sensor size scales depth of field
  • 2.5.10 Defocus is integration over the aperture dimension of the plenoptic function
• 2.6 Motion blur
  • 2.6.1 Blur from subject motion
  • 2.6.2 Blur from camera shake
  • 2.6.3 How shift-invariant is camera-shake blur?
  • 2.6.4 Which dominates: translation or rotation?
  • 2.6.5 The hand-holding rule of thumb
  • 2.6.6 Shake from the shutter and mirror
  • 2.6.7 Panning: fighting subject motion by moving the camera
  • 2.6.8 Freezing motion with light
  • 2.6.9 The space-time view: motion blur is integration over time
• 2.7 Sensors: photosites, CCD vs CMOS
  • 2.7.1 The photoelectric effect
  • 2.7.2 Photon to number: the photosite
  • 2.7.3 Beyond the visible spectrum: near-infrared, thermal, and ultraviolet
  • 2.7.4 Analog-to-digital conversion
  • 2.7.5 CCD versus CMOS
  • 2.7.6 Time integration: the exposure
  • 2.7.7 Shutters
  • 2.7.8 A taste of modern sensor tricks
  • 2.7.9 The impact of sensor size
  • 2.7.10 How far sensors have come
• 2.8 Noise, signal-to-noise ratio and dynamic range
  • 2.8.1 Noise sources
  • 2.8.2 The algebra of noise
  • 2.8.3 Signal-to-noise ratio: it's about ratios
  • 2.8.4 Noise variance is mostly affine in pixel brightness
  • 2.8.5 A fuller model of noise: Poisson photon noise plus clipping plus Gaussian electronic noise plus a second clipping
  • 2.8.6 Noise as a function of ISO
  • 2.8.7 Dynamic range
  • 2.8.8 Regimes, and how close we are to the limits
  • 2.8.9 Making noise variance uniform

▸ full collapsible outline of this part