31.3 Other light field acquisition setups🔗⧉

The plenoptic chip and the wall of cameras are the famous two, but they are not the only ways to record a 4-D light field, and for most of the field's history they were not even the common ones. Strip the instruments down to what they actually deliver and the requirement is modest: a light field is just the function $L(u,v,s,t)$, the radiance of each ray, indexed by where it crosses the aperture plane $(u,v)$ and the focal/sensor plane $(s,t)$, exactly as in Chapter 31.1, Light fields 101. Nothing in that definition says the rays must be captured at the same instant, or through one lens, or with special optics on the sensor. Any rig that can put a known ray onto a pixel, whether one ray now, another ray later, or several rays at once through a clever fold in the optical path, is sampling the field. So the design space is much larger than two points, and this chapter walks it.

31.3.1 One camera on a gantry: sample the aperture in time🔗⧉

The cheapest way to get many viewpoints is to own one camera and move it. Mount a single camera on a precise X–Y gantry, a motorized stage that translates it to programmed positions, and you can march it across a grid of viewpoints, firing one exposure at each. The grid of camera positions is the aperture sampling: each stop on the grid is one angular coordinate $(u,v)$, and the full-resolution frame taken there is that angular view. This is how Levoy and Hanrahan captured their original light fields in 1996 Levoy & Hanrahan 1996, and how the Stanford "gantry" rig sampled scenes at thousands of viewpoints, each a sharp full-frame image. Because every view comes off the whole sensor, there is no spatial-resolution tax at all. The gantry sits at the array end of the tradeoff, with the dense angular sampling of a plenoptic camera, simply paid for in time instead of cameras.

The catch is exactly that word, time. The viewpoints are captured one after another over seconds or minutes, so a static-field reconstruction requires a sufficiently static scene. Otherwise changes over capture time are mixed with viewpoint differences. For a still life or a museum object this is no obstacle, and the approach scales down: you do not need a Stanford-grade stage. A camera on a copy-stand, or a phone clamped to a hobbyist Lego rig that steps it across a plane, samples the same grid for a few dollars. The principle is indifferent to the price of the hardware: sample the aperture by moving the camera, not by subdividing pixels, which is exactly the principle the gantries, robotic arms, and flatbed-scanner rigs of Chapter 31.2, Light field cameras share.

A close cousin swaps which thing moves. Instead of carrying the camera around the scene, fix the camera and rotate the object on a turntable. Every rotation step presents a new face of the object to the lens, so the sequence of frames samples the rays leaving the object over a range of directions. Lighting must also remain consistent in object coordinates: rotating a glossy object under fixed room lights is not equivalent to moving a camera around a fixed, illuminated object. This outside-in capture is a natural choice for a single compact object you want to view from all around (the turntable variant of the same trade), and it generalizes to a spherical gantry, an arc or dome of camera positions that surrounds the subject. Turntables and spherical gantries are still the workhorses for digitizing museum artifacts and product models, where the subject sits patiently and a dense, all-around light field is worth the minutes it takes to sweep.

31.3.2 Handheld, unstructured capture: let the poses be irregular🔗⧉

A gantry buys you a regular grid of viewpoints, and regularity is convenient, because resampling the field for refocus or view synthesis is easy when the samples sit on a lattice. But the rigidity is also the cost: you need the rig. The next idea throws the rig away. Just wave a hand-held camera, or a phone, around the scene, shooting freely from wherever your hand happens to be. The viewpoints are now unstructured: irregularly spaced, unknown, scattered through space rather than placed on a grid.

The obvious objection is that you no longer know where each frame was taken, which was the main advantage of the regular grid, so you do not know which $(u,v)$ each one samples. The answer is to recover the poses after the fact. Run structure-from-motion on the captured images, the same multi-view-geometry machinery that powers photogrammetry, to estimate where the camera was for every shot, then resample the irregular bundle of rays onto whatever regular field you want to render from. This is the unstructured lumigraph of Buehler et al. Buehler et al. 2001: a light field captured with no special hardware at all, just a free-moving camera and the geometry to make sense of it afterward, with the view-blend weights set by each captured ray's angle, resolution, and field of view. A closely related line renders directly from the irregular bundle without committing to a single global proxy geometry, as in the unstructured light fields of Davis, Levoy and Durand Davis et al. 2012, which treat each hand-held image as a sample on a camera plane and reconstruct a novel view with a focal-stack-style sweep over candidate depths. Both trade the gantry's mechanical precision for a computational one, and they are the conceptual ancestors of the casual "wave your phone to capture a scene" workflows that show up in today's view-synthesis and neural-rendering pipelines.

31.3.3 Catadioptric capture: one sensor, many viewpoints at once🔗⧉

Every option so far pays in time: many exposures and a static scene. Can you get several viewpoints in a single shot without a microlens array? Yes: fold the optical path with mirrors. Aim one camera at the scene through a cluster of planar mirrors, a set of reflective facets, and each mirror presents the camera with a reflected view of the scene, as if a second camera were sitting behind the mirror looking out. These reflected cameras are virtual viewpoints: optically, the sensor sees the scene simultaneously from the real lens position and from each mirror's mirror-image position. The single sensor frame comes back tiled into several sub-views, one per facet, each looking at the scene from a slightly different direction, giving angular samples $(u,v)$ captured all at once, in one exposure, with no moving parts, but with the sensor’s pixels divided among views.

This is the catadioptric route, built by Han and Perlin in 2003 Han & Perlin 2003 with a tapered kaleidoscope, and echoed in mirror-ball and mirror-array rigs that scatter many reflections across one frame. Catadioptric means an optical system that uses both lenses (dioptric) and mirrors (catoptric). The tradeoff has simply moved again: you divide a fixed sensor area among viewpoints, trading per-view pixel count and often field of view. The sensor's image area is now divided among the sub-views, so each virtual viewpoint gets only a fraction of the frame: fine angular sampling and single-shot capture, bought with a smaller spatial extent per view and a limited number of viewpoints set by how many facets you can pack.

31.3.4 Coded aperture in time: sweep the pupil itself🔗⧉

The last idea keeps the single lens and the single sensor but adds a programmable mask at the main-lens aperture, re-patterned between exposures. The plenoptic camera sorted rays by angle with a microlens chip; here we sort them by selectively blocking the pupil. In the simplest scheme, each exposure opens only a small patch of the aperture and blacks out the rest, so that frame collects only the rays passing through that part of the pupil, which is exactly one angular sample $(u,v)$. Sweep the open patch across the pupil over a sequence of frames and you have visited every aperture position in turn, recovering a sampled 4-D field $L(u,v,s,t)$ and, because each frame still reads off the entire sensor, doing so at full spatial resolution, with no microlens.

This is the programmable-aperture capture of Liang et al. Liang et al. 2008. Like the gantry it pays in time: a static scene and one exposure per angular sample (the practical implementations use multiplexed, light-efficient patterns rather than a single open patch, then solve a small linear system to demultiplex the angular views, but the principle is the same sweep of the pupil). It sits squarely between the plenoptic camera and the gantry: a single, ordinary camera body that captures a full-resolution light field, at the price of needing several shots and a still subject.

A neighboring family replaces the microlens array with a mask while retaining the main imaging optics. Rather than block the pupil, place a fixed printed mask in the optical path and let it modulate the incoming rays, then untangle the angular information computationally from the single coded image. This is the heterodyne, or dappled-photography, approach of Veeraraghavan et al. Veeraraghavan et al. 2007: a patterned attenuating mask near the sensor casts the light field's angular structure into a frequency band that an ordinary sensor can record, and a frequency-domain decode pulls the 4-D field back out, giving a microlens-free, mask-based light-field camera. It is the same instinct as coded aperture (encode the rays, decode afterward) carried to the limit of "no lenslets, just a slip of printed film."

31.3.5 The everything-else, mapped🔗⧉

Step back and the zoo is not chaos; it is the same conservation law sampled from several sides. The plenoptic chip spends pixels; the camera array spends hardware; the gantry, turntable, and coded aperture spend time; the catadioptric kaleidoscope divides sensor area among views; the unstructured lumigraph still spends exposures and capture time, then pays additional computation (structure-from-motion plus resampling) to make irregular samples usable. Pick your currency and you have picked your rig. No design pays nothing, which is why the field has so many instruments rather than one winner, the same unresolved-tradeoff thread we follow into the frontiers chapter.

Two large branches of this zoo have grown up enough to get their own chapters, and we only point at them here so this one can stay the everything-else bin. Putting the microlens array not in a camera but at a microscope's intermediate image plane gives single-snapshot 3-D imaging of a specimen, namely light-field microscopy, taken up in Chapter 31.8, Light field microscopy. And putting a real array of cameras to work for high-speed capture, HDR, super-resolution, frozen-instant bullet-time sweeps, and the everyday multi-camera phone is the subject of Part 32, Multi-aperture imaging. Both are light-field acquisition in the strict sense; both earned a chapter; everything that did not is here. For the reader who wants the full survey of these instruments and the optics behind them, Ihrke et al. give a thorough account of the principles of light-field imaging Ihrke, Restrepo & Mignard-Debise 2016.