30.0 VIDEO⧉

A video adds a dimension the still image does not have: time. And time enters twice. A single frame is an integral over the exposure, so anything that moves while the shutter is open smears across the sensor: motion blur. And the clip as a whole is a sampling of time, so temporal frequencies above the sampling limit can alias: the wagon wheel that appears to spin backward. Once you can establish correspondence between frames, using the dense optical flow and sparse tracks built in Matching pixels across space and time, a whole family of temporal applications opens up: compress the clip, amplify motions too small to see, steady a shaky one, or synthesize frames that were never shot.

One technique in this part refuses to estimate motion at all, and the exception makes the pattern clear. Eulerian video magnification, the trick that makes a sleeping baby's pulse visible in an apparently still video, never asks where a pixel came from. It keeps each pixel location fixed and band-pass filters that location's brightness over time, then amplifies the small temporal variation: $I'(\mathbf x,t)=I(\mathbf x,t)+\alpha\,\mathcal B\{I(\mathbf x,t)\}$. By declining to track motion it sidesteps the entire ill-posed estimation problem. It can reveal small variations without estimating explicit trajectories, although noise, compression, and large motion still limit amplification. That is the Lagrangian vs Eulerian split: many other methods follow moving content; this one watches a fixed location's time series.

The chapters come next. Chapter 30.1, Motion blur, temporal sampling, and resampling sets up the time axis: blur as a time-integral, temporal aliasing, and why the two are one tradeoff. Then the applications, each one correspondence-then-transport in a particular key. Chapter 30.2, Video compression and motion compensation is correspondence on a budget: predict each block from the previous frame along a motion vector, then code only the cheap residual; optical flow wearing the codec's constraints. Chapter 38.3, Motion and video magnification is the Eulerian reveal, the deliberate refusal to estimate motion. Chapter 30.6, Video stabilization and rolling-shutter correction is the full pipeline: estimate the shaky camera path, smooth it, re-render every frame against the smoothed path (and unjellify the rolling-shutter wobble while you are at it). Chapter 30.4, Frame interpolation and slow-motion synthesis transports pixels to the in-between moment, flowing both neighbors toward a time that was never captured. Chapter 30.3, Video editing moves to the timeline, covering summarization, reduce-over-time filters, transcript-based cutting, and frame-synthesizing in-betweening, where motion understanding meets the cut. And a closing chapter, Chapter 30.8, Video textures, folds the time axis back on itself, re-ordering a short clip's own frames into an endless seamless loop, the temporal cousin of the panorama seam.

Contents of this part

▸ full collapsible outline of this part

30.0 VIDEO🔗⧉

Contents of this part

30.0 VIDEO⧉