💬Comments welcome. To leave a note, select any text and click the note / highlight button that pops up — or open the panel with the tab at the top-right (‹). Notes are visible only inside our private review group.
jump to

12.0 VIDEO

A video adds a dimension the still image does not have: time. And time enters twice. A single frame is an integral over the exposure, so anything that moves while the shutter is open smears across the sensor — motion blur. And the clip as a whole is a sampling of time, so motion faster than the frame rate folds back as temporal aliasing — the wagon wheel that appears to spin backward. Once you can establish correspondence between frames — the dense optical flow and sparse tracks built in Matching pixels across space and time — a whole family of temporal applications opens up: compress the clip, amplify motions too small to see, steady a shaky one, or synthesise frames that were never shot.

One technique in this part refuses to estimate motion at all, and the exception is so clean it proves the rule. Eulerian video magnification — the trick that makes a sleeping baby's pulse visible in an apparently still video — never asks where a pixel came from. It fixes its gaze on each pixel location and band-pass filters that location's brightness over time, then amplifies the tiny temporal wiggle: $I'(\mathbf x,t)=I(\mathbf x,t)+\alpha\,\mathcal B\{I(\mathbf x,t)\}$. By declining to track motion it sidesteps the entire ill-posed estimation problem — and gets to amplify motions far too small to track. That is the Lagrangian vs Eulerian split in a nutshell: every other chapter here follows the moving content; this one watches a fixed location's time series.

The roadmap. Motion blur and temporal sampling sets up the time axis — blur as a time-integral, temporal aliasing, and why the two are one tradeoff. Then the applications, each one correspondence-then-transport in a particular key. Video compression and motion compensation is correspondence on a budget — predict each block from the previous frame along a motion vector, then code only the cheap residual; optical flow wearing the codec's constraints. Video magnification is the Eulerian reveal, the deliberate refusal to estimate motion. Video stabilization and rolling-shutter correction is the full pipeline — estimate the shaky camera path, smooth it, re-render every frame against the smoothed path (and unjellify the rolling-shutter wobble while you are at it). Frame interpolation and slow-motion synthesis transports pixels to the in-between moment, flowing both neighbours toward a time that was never captured. And a closing Video editing coda steps up to the timeline — summarisation, reduce-over-time filters, transcript-based cutting — where motion understanding meets the cut.


Contents of this part

▸ full collapsible outline of this part