32.3 Multi-camera phones⧉
The arrays of Chapter 32.1, Camera arrays: one rig, many instruments are bulky lab rigs and the cautionary L16 was a brick. But the array idea did escape into the wild, shrunk to the device in your pocket. You almost certainly own a camera array. Look at the back of a modern phone and you find a small cluster of cameras, each with a different focal length, typically an ultrawide, a wide (the main camera), and a telephoto, sometimes a folded periscope tele for longer reach. These are usually separate, fixed-focal-length lenses sitting a centimeter or two apart, and the phone's job is to make them feel like one continuous camera. It does this by fusing them.
Two kinds of fusion matter. The first is across the zoom range. As you pinch to zoom, the phone switches from one module to another and blends the handoff so the transition feels smooth, and in the overlap it can borrow detail from whichever module is sharper at that magnification. Where illumination, focus, and overlap permit, the tele can supply finer detail than a digital crop of the wide, so the wide's frame is sharpened toward it. The second kind of fusion is for depth. Two modules a known distance apart see the scene from two slightly different viewpoints, so the disparity between them is a stereo cue: near objects shift more between the two images than far ones do. From that disparity the phone estimates a depth map, and that depth map drives portrait mode, the synthetic, depth-dependent background blur of Chapter 23.2, Fake (synthetic) depth of field, applied computationally to extend or control the optical defocus the phone already produces. Phones can produce real optical background blur, especially at close focus. Wadhwa et al. 2018 specifically combines learned segmentation with dual-pixel disparity, so it is not solely an example of stereo between separate modules.
One caveat matters here: this is discrete optical steps plus computational interpolation, not a continuous optical zoom. A real zoom lens slides its elements to cover every focal length in a range optically. The common multi-module design has discrete focal lengths and computes intermediate settings; some phones also contain a variable optical telephoto module. At an intermediate "zoom" you are looking at a cropped, upscaled, detail-transferred composite, not light optically magnified to that exact setting. The seams are usually invisible, which is the achievement, but the underlying reality is a handful of fixed lenses stitched in software.
And the same parallax that gives you depth also costs you alignment. Because the modules sit a few centimeters apart, they do not see the same image: near objects sit in different places in the ultrawide and the tele, foreground can occlude background differently in each, and a glossy highlight can appear in one module and not another. All of that disparity between modules must be reconciled in software before the frames can be fused, or the blend ghosts at exactly the depth edges where, as the focal-stack chapter warned, occlusion makes the right answer flip abruptly. The phone array is the everyday case of the whole part's tension: parallax is simultaneously the signal (depth) and the nuisance (mis-registration). That is the same bargain Wilburn's hundred-camera grid struck, now in a phone you carry every day.
