32.0 MULTI-APERTURE IMAGING⧉
The previous part dissected a single aperture. Record the rays that cross one lens, finely diced into angular samples by a microlens array, and any ordinary photograph becomes a computation you run afterward: the plenoptic camera of Part 31, Light fields and plenoptic cameras. This part takes the same light field from the other end. Instead of one aperture sliced finely, use many whole apertures, a rig of full cameras with each sensor at full resolution, and fuse their frames computationally. Same 4-D light field, opposite bargain: the plenoptic camera buys fine angle and pays in spatial resolution; the array keeps every pixel and pays instead in bulk, calibration, and bandwidth.
That is the central theme of the part. Once a grid of synchronized cameras exists, it is a programmable instrument, and the instrument is configured through trigger timing and exposure before capture, then through fusion afterward. Stagger the triggers and the array becomes a high-speed camera running at $N$ times any one sensor's frame rate. Vary the exposures and it captures high dynamic range in a single instant, no bracketing in time. Exploit the sub-pixel offsets between viewpoints and it feeds super-resolution. Align the views to a depth plane and sum them and you have synthesized one enormous aperture so shallow it sees through foliage, fences, and crowds. Step through the views instead of summing them and a single frozen instant can be swept in viewpoint: Chapter 32.2, Bullet time, with the subject hanging motionless while the camera orbits. And the everyday case sits in your pocket: Chapter 32.3, Multi-camera phones, a cluster of fixed lenses fused into something that feels like one continuous camera and reads its own parallax as portrait-mode depth.
That pocket case matters commercially; it is the most consequential multi-aperture system ever built. A phone cannot grow a bigger lens or a deeper sensor without getting thicker, so to compete with "real" cameras despite being millimeters thin it goes sideways instead of bigger: several small modules side by side, ultrawide, wide, and tele, fused in software into a system that outperforms any single module, spanning a range of focal lengths, gathering more light, and reading depth for portrait mode, within the phone’s packaging constraints, often including a camera bump. Multi-aperture imaging is one of the defining moves of the computational-photography era on phones. The idea predates the smartphone, too: Kodak's EasyShare V570 (2006) already carried two lenses, an ultra-wide paired with a normal one, the first dual-lens consumer digital camera, and its V610 followed the same year with two overlapping telephoto lenses for a longer combined zoom, a full decade before dual cameras became standard on phones.
One tension should be introduced before the chapters begin. Separate apertures see slightly different scenes: the parallax between them. That disparity is at once the signal and the cost: it is depth (near objects shift more than far ones), and it must be undone before frames from different viewpoints can be fused without ghosting at exactly the depth edges where occlusion makes the right answer flip. Every chapter here pays that tax and collects that dividend, from a hundred-camera grid down to the three modules behind a phone.
The part runs from the rig to the pocket:
Chapter 32.1, Camera arrays: one rig, many instruments takes the synchronized grid seriously as an instrument, using synthetic-aperture see-through, high-speed, HDR, and super-resolution as four fusions of the same frames, and closes on the commercial array, the Light L16, as the cautionary version of fusion at scale. Chapter 32.2, Bullet time separates out the angular use of the array: a ring of cameras fired at one instant, played back in spatial order to sweep a frozen moment. Chapter 32.3, Multi-camera phones follows the idea down to the device you own: ultrawide/wide/tele fusion across discrete optical steps plus computational interpolation (not a continuous zoom), and the inter-module disparity that yields portrait depth.
The part reduces to one claim: the plenoptic camera dices one aperture finely; the array fuses many whole apertures coarsely, and the same parallax that costs you alignment is the depth you get back. It is one continuous idea, scaled from a single sensor split in two (Chapter 22.9, Autofocus, the smallest possible light field) up to Wilburn's grid of a hundred: record many apertures and reconcile them afterward.
Contents of this part
- 32.1 Camera arrays: one rig, many instruments
- 32.1.1 One rig, four instruments
- 32.1.2 Commercial arrays: the Light L16
- 32.1.3 Where this sits
- 32.2 Bullet time
- 32.3 Multi-camera phones