• **continuous-wave (phase) ToF** — illuminate the scene with amplitude-modulated light and measure the **phase shift** of the returning signal per pixel → range (e.g. **Kinect v2**); ambiguity beyond one modulation wavelength is resolved with multiple frequencies. Vs **direct / pulsed ToF** (LiDAR, above): time an actual pulse. CW-ToF is cheap and dense but suffers **multipath** (light bouncing twice reads as wrong depth); pulsed/SPAD ToF is longer-range and multipath-robust.

• a per-pixel **timing** measurement — the same physics as the SPAD direct-ToF below, on the modulation side rather than single-photon counting.

• **color is a projection** (recap, [[Visual processing]]): a camera channel integrates the incoming spectrum $S(\lambda)$ against a **spectral sensitivity** $R_c(\lambda)$: $I_c=\int S(\lambda)R_c(\lambda)\,d\lambda$. RGB uses **three broad, overlapping** $R_c$ → three numbers. That projection is **lossy and many-to-one**: distinct spectra can give **identical** RGB (**metamers**). [`fig-hyperspectral-rgb-vs-spectrum`]

• **what's lost matters**: two pigments, two plant species, fresh vs spoiled produce, a forgery's ink vs the original — can look identical to RGB yet have **different spectra**. The information needed to tell them apart is *exactly* what RGB discarded.

• **the generalization**: keep the same equation but use **many narrow** band responses $R_b(\lambda)$ (approaching deltas at wavelengths $\lambda_b$). Now each pixel yields a **sampled spectrum** $S(x,y,\lambda_b)$ — the **hyperspectral data cube** $I(x,y,\lambda)$, an image stack whose third axis is **wavelength**. **RGB is just this cube with 3 broad bands** (and a CFA/Bayer mosaic is a 3-band snapshot mosaic — cross-ref [[Measuring and encoding color|Color technology]]). [`fig-hyperspectral-cube`]

• **multispectral vs hyperspectral**: *multispectral* = a handful of bands (e.g. Landsat's ~7, or RGB+NIR); *hyperspectral* = many **contiguous, narrow** bands (tens–hundreds) → a quasi-continuous spectrum per pixel. Same idea, different sampling density along $\lambda$.

• **the unifying view (L11.2)**: each method **fills the cube** $I(x,y,\lambda)$ differently, trading **time vs spatial vs spectral** resolution — the spectral analog of "how do you capture the focal/exposure stack." [`fig-hyperspectral-capture`]

• **spectral scanning — filter wheel / tunable filter**: put a **bandpass filter in front of the sensor** and capture **one wavelength band at a time**, then change the band:

• a mechanical **color/filter wheel** rotates discrete narrow filters into place;

• an electronically **tunable filter** — **LCTF** (liquid-crystal) or **AOTF** (acousto-optic) — sweeps the passband with **no moving parts**, fast and programmable.

• tradeoff: **full spatial resolution**, many bands, but **sequential in time** → the **scene (and camera) must hold still** across the sweep. Great for microscopy, lab, art on an easel.

• **spatial scanning — pushbroom (line-scan)**: image a **single line** of the scene through a **slit + dispersing element** (prism/grating), so the sensor records that line spread across **all wavelengths at once** (one sensor axis = position along the line, the other = wavelength). **Sweep the line** across the scene (platform motion) to fill $(x,y)$:

• natural when the scene already moves past the sensor — **satellites/aircraft** (the ground sweeps beneath), **conveyor belts** (food/mineral sorting).

• tradeoff: **all bands captured per line simultaneously**, but needs **relative motion** and good line-to-line registration.

• **snapshot spectral imaging**: capture the whole cube in **one exposure** — e.g. a **spectral filter mosaic** (a Bayer-like array but with many band filters) or computational/coded designs (Hagen & Kudenov 2013). Tradeoff: **one shot (handles motion/video)** but **trades spatial resolution** for spectral bands (the mosaic spends pixels on wavelength).

• **coded snapshot / CASSI — the inverse-problem version** (Wagadarikar et al. 2008): the most compelling snapshot designs are **computational** — a **coded aperture + a dispersive element** (prism/grating) optically **multiplex** the full $(x,y,\lambda)$ cube onto a *single* 2-D sensor frame. The coded mask shears and scrambles the bands so each sensor pixel sums a *coded mixture* of wavelengths; the cube is then **recovered by solving a compressive-sensing inverse problem** — find the $(x,y,\lambda)$ cube whose coded projection matches the measured frame, regularized by a **sparsity / learned prior** (the spectrum is sparse in some basis, or a learned spectral prior fills in the rest). This is **snapshot compressive spectral imaging (CASSI)**: you *trade a reconstruction for a single-shot capture* — one frame instead of the slow filter-wheel/pushbroom **scan**, at the cost of solving an under-determined inverse problem. It is the same coded-imaging / compressive-sensing move as coded-aperture and single-pixel imaging — cross-ref **compressive sensing / coded imaging** in [[Computational optics and coded imaging]].

• **the recurring tradeoff triangle**: you can have any two of {full **spatial** resolution, full **spectral** resolution, **single-shot/fast**} cheaply, but not all three — spectral scan sacrifices time, pushbroom needs motion, snapshot sacrifices spatial resolution.

• **material identification / unmixing**: a pixel's **spectrum is a fingerprint**; match it to a library to **classify materials** (minerals, plastics, vegetation species), or **unmix** a pixel that is a blend of several materials. The core remote-sensing use (imaging spectrometers like AVIRIS). [`fig-hyperspectral-uses`]

• **agriculture / vegetation**: plants reflect strongly in **near-infrared**; indices like **NDVI** $=\frac{I_{\mathrm{NIR}}-I_{\mathrm{red}}}{I_{\mathrm{NIR}}+I_{\mathrm{red}}}$ reveal vigour, water stress, disease before it's visible — a per-pixel **band ratio** (a normalized, multiplicative index — L2.6, the multiplicative-world lesson). Precision agriculture, crop monitoring.

• **art conservation & forensics**: reveal **underdrawings, pentimenti, retouching**, identify **pigments**, and detect **forgeries** by spectral differences invisible to the eye / to RGB; document and authenticate.

• **other**: **food inspection** (ripeness, contamination), **medical/biological** imaging (tissue oxygenation, fluorescence microscopy with LCTF/AOTF), **recycling/sorting**.

• **closing tie**: in every case the discipline is L11.2 — **record the full spectral set, then ask the question** (which material? healthy or stressed? original or forgery?) long after capture. The wavelength axis joins exposure (HDR), focus (focal stacks) and aperture (light fields) as another axis you can defer.

• **three things light carries**: **how much** (intensity), **what color** (spectrum — the wavelength axis of [[#Hyperspectral imaging, color wheels]]), and **how it oscillates** (polarization — the orientation of the electric field). A normal sensor keeps the first, three-band-samples the second, and is **blind to the third**. [`fig-polarization-axis`]

• **why it's informative**: light becomes **partially polarized** when it reflects off a **dielectric** (glass, water, leaves, skin), scatters in the **atmosphere** (the blue sky is strongly polarized in a band 90° from the sun — bees and ants navigate by it, Wehner), or glances off a wet road. **Specular** (surface) reflection is strongly polarized; **diffuse** (body) reflection that has bounced around inside a material is largely **depolarized**. That asymmetry — *surface light is polarized, body light isn't* — is the lever every technique in this chapter pulls.

• **the discipline (L11.2)**: you can't see polarization in one ordinary frame, so **capture a stack** of it and decide later, exactly as for exposure and wavelength.

• **the unifying view**: each method **fills the polarization stack** $I(x,y,\theta)$ differently, the same **time vs spatial-resolution** trade as hyperspectral's scan-vs-snapshot. [`fig-polarization-mosaic`]

• **rotate a linear polarizer**: put a polarizer on the lens and shoot at **several angles** — classically **0°, 45°, 90°, 135°** (three suffice; four is robust). Per pixel, intensity traces a **cosine in $2\theta$** (Malus's law) whose **offset, amplitude and phase** give the total intensity, the **degree** and the **angle** of polarization. Full spatial resolution, but **sequential** → the scene and camera must hold still (registration as in [[#Denoising by averaging]]).

• **division-of-focal-plane (DoFP) sensor — the snapshot route**: a **micro-polarizer mosaic** (0/45/90/135° wire-grid filters in a $2\times2$ super-pixel) sits over the photosites, a **Bayer-like array for polarization** — Sony's **IMX250MZR**. One exposure yields all four angles, at the cost of **spatial resolution** (and demosaicing). This is the polarization instance of **assorted pixels** — spend resolution to buy a new axis — developed on the sensor side in [[Assorted pixels]].

• **recovering the Stokes parameters**: from the per-angle intensities, $s_0=I_0+I_{90}$ (total), $s_1=I_0-I_{90}$, $s_2=I_{45}-I_{135}$; then **DoLP** $\rho=\sqrt{s_1^2+s_2^2}/s_0$ (how polarized, 0–1) and **AoP** $\phi=\tfrac12\operatorname{atan2}(s_2,s_1)$ (the oscillation angle). These two scalar maps are what the combines below operate on. [`fig-polarization-axis`]

This is the heart of the chapter — the **merge** step, the polarization analog of stitching a bracket into a radiance map. Because specular and partially-scattered light are polarized while diffuse object light is not, the *right combination* of the polarized frames **isolates or removes** a component.

• **reflection / glare removal** (Nayar, Fang & Boult 1997): shoot through glass or at a glossy surface and the **specular reflection is strongly polarized**, the **scene behind it largely not**. Combine the frames to **subtract the polarized component** — the window reflection, the glare on a painting, the sheen on a leaf fades, revealing the diffuse image underneath. At **Brewster's angle** the suppression is near-total; this is the photographer's circular-polarizer trick made **per-pixel and computational**, and the passive cousin of [[Computational illumination|reflection removal]]. [`fig-polarization-reflection-removal`]

• **specular–diffuse separation**: the same split, used not to *remove* but to *separate* — the polarized (varying) part is the **surface/specular** layer, the unpolarized offset the **diffuse/body** layer. Feeding the clean diffuse layer to color constancy, stereo or recognition removes the specular outliers that break them.

• **dehazing and underwater descattering** (Schechner, Narasimhan & Nayar 2001; Schechner & Karpel 2005): in haze or water, the veiling **airlight/backscatter is partially polarized** while the object's light is less so. From **two orthogonally-polarized frames** you estimate the **airlight magnitude and transmission**, then invert the scattering model $I = t\,L + (1-t)A_\infty$ to recover a **descattered, higher-contrast** image with restored depth — "instant dehazing" from just two frames, no scene model needed. [`fig-polarization-dehaze`]

• **shape from polarization** (Atkinson & Hancock 2006; Kadambi et al. 2015): the **AoP** is tied to a surface's **azimuth** and the **DoLP** to its **zenith angle**, so the polarization stack yields a dense field of **surface normals** — a passive shape cue. It is ambiguous alone (the azimuth has a π-ambiguity, the diffuse/specular regime matters), so the strong modern result **fuses it with a coarse depth map** (ToF or stereo) — **Polarized 3D** — to disambiguate and **upgrade low-res depth to high-quality normals**. A nice instance of combining a cheap cue with a rich one. [`fig-shape-from-polarization`]

• **the closing tie**: every one of these is L11.2 — **capture the polarization set, then ask the question** (glare or surface? how far through the haze? which way does it face?) per pixel, long after the shutter. Polarization joins exposure (HDR), wavelength (hyperspectral) and aperture (light fields) as another axis of the plenoptic signal you can defer and recombine.