31.5 Aberration correction in light fields🔗⧉

This is the less-known and possibly most important chapter of Ren Ng's thesis (Ng 2006), and it follows from a single observation. Lens designers spend their careers fighting aberrations, the ways a real lens fails to focus a point to a point, sending rays from the same scene point to slightly different places on the sensor and smearing what should be sharp. The optical chapter, Chapter 22.2, Aberrations correction, met these as a problem of glass: add elements, choose exotic dispersions, asphere a surface, and bend each ray back toward where it belongs. That is hard, heavy, and expensive, and it is the burden every camera lens carries.

Now recall what a light-field camera actually records. An ordinary pixel sums all the rays crossing the aperture, so once a ray has landed in the wrong place its angular identity is lost in that measurement; image-domain deconvolution can recover some contrast only where the optical transfer and noise permit. But a light-field camera keeps each ray separately. It records, for every sampled ray, which ray it was and where it landed. And that reframes the whole problem. A lens aberration is simply rays landing at the wrong place. If you know the lens, that is, the map from a ray's entry through the aperture to where the real lens deposited it, then you also know where an ideal lens would have deposited it. So you can take each recorded ray and re-route it to the ideal-lens position, correcting the aberration computationally, from one capture.

That is the whole chapter. We will state the correction precisely, place it against the optical fix of Chapter 22.2, Aberrations correction and against deconvolution, and be candid about what it costs.

31.5.1 An aberration is misrouted rays🔗⧉

Start from the ideal. A perfect lens maps every ray leaving a scene point, whatever part of the aperture it passed through, onto the same image point; that is what "in focus" means. A real lens does not. Rays through the outer aperture may bend too strongly (spherical aberration), rays of different colors land at different distances (chromatic aberration), off-axis points smear into comet shapes (coma), and so on. In every case the geometry is the same: rays from one scene point arrive at a spread of image positions instead of one, and the sum of that spread is the blur you see. The optical chapter fights this spread by reshaping the glass so the rays converge again. The light-field camera instead lets the rays land wherever the real lens sends them, but it writes down where each one went.

This is the move the whole part is built on. A light field tags every recorded ray by where it crossed the two planes (its aperture coordinate and its image coordinate), so the data is a function over rays, not a single blurred image. The aberrated capture is not a smear we have to interpret; it is a record of rays, each entry saying "this ray, from this part of the aperture, landed here."

31.5.2 Re-routing each ray to the ideal-lens position🔗⧉

The correction is then exactly what it sounds like. We know the real lens: from its design (or a calibration) we know, for a ray entering at a given aperture position and angle, where the real lens actually put it, the address we recorded. We also know where the ideal lens would have put that same ray. So for every recorded ray we have both its measured landing spot and its ideal landing spot, and we simply re-project the ray from the measured position to the ideal one, moving its recorded value to the position an aberration-free lens would have given it (Figure 31.5.1⛶ fullscreen). Do this for every sampled ray and the rays that the real lens scattered across a smear are gathered back to the single point they should have formed. The aberrated capture becomes a corrected image.

This is mechanically the same re-projection as refocusing in Chapter 31.4, Refocusing and synthetic aperture. Refocusing re-routes each recorded ray by the shift that brings a chosen depth into focus and sums; aberration correction re-routes each recorded ray by the lens-dependent displacement that an ideal lens would have produced, and sums. Both are pure ray bookkeeping on the recorded light field: a re-projection, not a deconvolution. With calibrated, sufficiently resolved ray samples, this places measured contributions at corrected coordinates rather than deconvolving an aperture-integrated blur.

31.5.3 The trade: ray bookkeeping instead of glass, and only what you sampled🔗⧉

The plain accounting is a trade, and it is the trade that runs through this whole part. You swap the optical-design burden for ray bookkeeping. The optical fix of Chapter 22.2, Aberrations correction uses more glass, with more elements, tighter tolerances, exotic materials, weight, and cost, to make the rays converge in hardware. The light-field fix uses more data: you must record the rays in the first place, which means a microlens array and the plenoptic resolution cost, namely the spatial resolution you spent on angular samples back in Chapter 31.2, Light field cameras. A simpler, cheaper, more aberrated lens becomes acceptable because its sins are corrected in software, but the camera that records the rays gives up resolution to do it.

There is also a hard limit, and it is the limit of every measurement: you can only correct what you actually sampled. The re-projection moves the rays you recorded; it can do nothing for rays the camera never captured. So the correction is bounded by the aperture rays you sampled, that is, by the angular sampling of the microlens array, and detail carried by rays outside that sampling, or finer than the angular grid resolves, is simply not in the data to be re-routed. Within those bounds, though, the contrast with the deblurring view is clean. A deconvolution / PSF approach (the inverse-problem view of Chapter 6.7, Image formation as a linear system) treats the aberrated image as a blur and tries to undo it, estimating the lost detail back and paying in noise amplification and instability. The light-field correction does not invert a blur at all: it never lost the geometry, because it recorded which ray went where, so it just puts each measured ray in its right place. You correct a calibrated geometrical mapping directly, but finite spatial and angular cells have already integrated light. Diffraction, unresolved variation within a cell, calibration errors, and chromatic integration can still limit the correction, which is why this idea may be, as Ng suggests, the most important one in the thesis (Ng 2006).