32.1 Camera arrays: one rig, many instruments⧉
The plenoptic camera packs its angular samples inside one lens, and pays for them in spatial resolution: fine angle, coarse image. Push to the other end of the spectrum and you get the opposite bargain: a grid of synchronized, full-resolution cameras spread over a baseline Wilburn et al. 2005. Each camera is an ordinary sensor at full pixel count, so you keep all the spatial resolution; what you spend instead is bulk, calibration, and bandwidth. The interesting part is what that grid becomes once it exists. A camera array is a programmable instrument, and almost everything in this part drops out of it by changing only one thing: how you fire the cameras and how you fuse their frames.
The same physical rig, with the Stanford array of Wilburn et al. 2005 as the canonical example, can be wired for at least four different jobs, and the array end of the light-field spectrum is most useful seen this way: as many apertures, fused computationally, where capture timing and exposure prepare the data for the fusion you later run.
32.1.1 One rig, four instruments⧉
Start with the operation we already have. Synthetic aperture was developed in Chapter 31.4, Refocusing and synthetic aperture: align all the views to a chosen depth plane and average them, and you have synthesized a single lens as wide as the whole rig. Because that synthetic aperture is enormous, its depth of field is razor-thin. That is what lets it see through occluders Levoy et al. 2004. Foliage, a fence, or a crowd in front of the subject is so far off the focus plane that it smears into a faint haze when the views are summed, while the subject, sharp on the plane, resolves cleanly. The same shift-and-add also refocuses the array after capture, just as it did for the single light field. So the array inherits, for free, everything the refocusing chapter built; here it simply runs at full resolution over a wide baseline.
Now keep the rig and change the firing. Three more uses follow.
High-speed video. If the cameras share a clock but you stagger their triggers, with camera 1 firing, then camera 2 a fraction of a frame later, and so on around the grid, a single frame-time produces a whole sequence of slightly offset exposures. Interleave those staggered frames in time order and you have a video at $N$ times the frame rate of any one camera. A $10\times10$ array of 30-fps cameras can provide 3000 staggered samples per second. To resolve motion at that rate, each exposure must also be short enough; long overlapping exposures do not become crisp merely because their start times are staggered. The catch is that successive frames come from different viewpoints, so the parallax between cameras must be undone before the frames can be read as a single high-speed clip. That alignment step recurs throughout the part.
High dynamic range. Give each camera a different exposure, some short and some long, and one synchronized snapshot captures the scene at many exposure levels at once. Merge them per pixel and you have an HDR image with no bracketing in time, with overlapping capture times. Different exposure durations still integrate moving content differently, and separate viewpoints introduce occlusion and registration problems. This is exactly the exposure-axis lesson of Multiple exposure imaging, but paid for in cameras rather than in time: instead of capturing the bracket sequentially and assuming the scene stays still, you capture it all at once across the grid.
Super-resolution. Because the cameras sit at slightly different positions, their registered samples can have complementary sub-pixel phases, although raw disparities may be many pixels and vary with depth. Several full-resolution views that are each offset by a fraction of a pixel carry, between them, more spatial detail than any one of them alone. The missing high frequencies are hidden in the inter-view shifts. Combining them into a single higher-resolution image is the super-resolution problem of Chapter 14.2, Multi-frame super-resolution, and the array is an ideal front end for it: many real, simultaneous, sub-pixel-shifted samples rather than the incidental hand motion a single camera relies on.
Note what unifies the four. None of them changes the hardware. You build one grid of synchronized full-resolution cameras, and then synthetic aperture, high-speed, HDR, and super-resolution are four uses of the same hardware with appropriately configured captures, chosen by how you set the triggers and exposures and by which merge you run afterward. The array is the array; the instrument is software. (A fifth use, bullet time, exploits the array for its angular dimension rather than its aperture; it gets its own chapter next, Chapter 32.2, Bullet time.)
32.1.2 Commercial arrays: the Light L16⧉
If a phone is a three-camera array, the subject of Chapter 32.3, Multi-camera phones, the natural question is how far you can push it, and one company tried to push it hard. The Light L16, now defunct, packed sixteen small camera modules of mixed focal lengths into a body the size of a thick phone, selected several modules for a shot and fused their frames into a single high-resolution image with a computed depth map. The idea was that many cheap small modules, fused, could stand in for one large lens and sensor.
The L16 matters here because it embodies both the promise and the difficulty of array fusion. The promise is real: many apertures genuinely carry more information than one, and in principle you can trade lens bulk for computation. The difficulty is also real, and it is the parallax problem at scale: sixteen modules at sixteen viewpoints and several focal lengths must all be registered and merged into one coherent image, and any error in that reconciliation shows up as artifacts at depth edges. The L16 did not survive as a product. It stands as the cautionary version of this part: the fusion that a phone pulls off invisibly across three modules becomes markedly harder across sixteen.
32.1.3 Where this sits⧉
Step back to the spectrum the light-field part built. At one end is the plenoptic camera of Chapter 31.2, Light field cameras: one body, one shutter, fine angular sampling, low spatial resolution, with the whole light field captured hand-held in a small device. At the other end is the camera array: a rig of full-resolution sensors over a wide baseline, full spatial resolution and a long enough baseline to see through occluders, but bulky, hard to calibrate, and heavy on bandwidth. Multi-aperture imaging is what you do once you commit to the array end and treat it as a programmable instrument: synthetic aperture, high-speed, HDR, super-resolution, bullet time, and the phone cluster are all points along it.
And the spectrum does not stop at the plenoptic camera; it runs all the way down to two sub-apertures, then to one split in half. Dual-pixel autofocus, with every photosite divided into two half-pixels that each see one half of the lens aperture, is the smallest possible light field: a two-sample, two-view capture from a single ordinary sensor, used in Chapter 22.9, Autofocus to measure defocus disparity and drive focus, and the same tiny disparity can be read as depth. From one body split in two, to a phone's three modules, to Wilburn's grid of a hundred, it is one continuous idea: record many apertures and reconcile them afterward. The number of apertures changes; the bargain, more views, more parallax to undo, and more to recover, does not.