15.5 Alternative color filter arrays⧉
Every color filter array trades the same three things against each other: how much light reaches the sensor, how finely luminance is sampled, and how well color can be reconstructed. Bayer's answer has held for fifty years because it is a good compromise, not because it is optimal, and the alternatives are all legible as different weightings of those three.
Two properties of Bayer are the baseline for the comparisons below. Half the pixels are green, placing green on a quincunx lattice. Luminance-like information is reconstructed using all channels, not measured only by green. And the pattern is periodic with a $2 \times 2$ tile, which makes the modulation structure simple and the reconstruction well understood.
15.5.1 Fuji's X-Trans⧉
X-Trans uses a $6 \times 6$ tile with a quasi-random arrangement, containing more green than Bayer and, crucially, at least one pixel of every color in every row and column.
X-Trans aims to reduce objectionable color moire with a different arrangement of color samples, allowing cameras to omit an optical low-pass filter in their design. The repeating six-by-six tile is still periodic. It redistributes modulation components rather than eliminating aliasing or guaranteeing noise-like artifacts. The pattern also needs its own demosaicking: a converter tuned for Bayer cannot be reused, and some raw converters, when X-Trans was new, rendered fine foliage and grass with a smeared, wavy texture that photographers called "worms". The artifact came from the reconstruction, not from the array, and it diminished as converters improved.
Its different local sampling requires a demosaicker designed for that layout. The longer repeating tile has a well-defined Fourier-series decomposition, with more carrier locations than Bayer. Frequency-domain analysis still applies. Evaluate fine texture and false color using the actual converter rather than assuming an intrinsic ranking against Bayer.
This is also a hardware/software coordination problem. A sensor designer can choose a pattern with attractive optical or sampling properties, but the resulting raw files still need support in the photographer's converter. The quality seen by the user is the quality of that complete combination. Comparing sensors through converters that handle their layouts differently can therefore confuse a reconstruction limitation with a sensor limitation.
15.5.2 Quad-Bayer and its modes⧉
Quad-Bayer groups four adjacent pixels under one color filter, giving a $4 \times 4$ tile that is a Bayer pattern of $2 \times 2$ blocks. Related grouped-color layouts are common in high-resolution phone sensors, and it exists because the same sensor has to serve two contradictory requirements.
Binning combines same-color samples to produce a coarser mosaic. Summing four equal-exposure samples quadruples signal and doubles independent shot-noise standard deviation, giving twice the shot-limited SNR at lower spatial resolution. Read-noise behavior depends on whether combination occurs before or after readout. A sum and an average also use different signal units.
Nona-Bayer extends the idea to $3 \times 3$ blocks, and the largest phone sensors use $4 \times 4$. At full resolution a grouped mosaic cannot go to a Bayer demosaicker as it is. Phones usually remosaic it, estimating a conventional Bayer mosaic that the existing image signal processor accepts, which is itself an interpolation; the alternative is to demosaic the grouped layout directly. Chapter 15.10, Quad-Bayer and Nona-Bayer processing compares the two routes and the methods for each.
Split exposure. Give the four pixels in a group different exposure times or different conversion gains, and a single capture contains both a bright and a dark read of nearly the same location, which extends dynamic range without bracketing. The cost is that the four pixels no longer measure the same thing, so binning and remosaicking both have to account for it.
The manufactured CFA remains fixed; the readout mode changes how its samples are combined or reconstructed. Preserve both layout and mode metadata.
15.5.3 RGBW and sparse color⧉
An RGBW array includes clear pixels with a broader response than its color-filtered pixels. Their photon gain depends on the filter spectra, illumination and sensor response; it is not universally a factor of three.
A clear pixel measures a broad spectral integral, not calibrated luminance directly. Reconstruction combines that signal with fewer color-selective measurements. Infrared leakage, illuminant changes and sharp chromatic structure affect how well the clear response predicts visible brightness.
Sparse color arrays push this further, using fewer color-selective samples alongside denser broad-response measurements. Their motivation is that spatial sensitivity to chromatic variation is often lower than sensitivity to luminance variation (Chapter 4.5, Advanced color: opponents, perceptually-uniform, and preferred). They can work well where color varies slowly, but sharp chromatic patterns expose the sparse color sampling.
Every array in this chapter still samples color in space. The alternatives that sample it in time, in silicon depth, or in extra bands are Chapter 15.12, Hardware alternatives to the mosaic, and one of them, stepping the sensor by exactly one pixel between four exposures, removes demosaicking from the problem entirely for a scene that holds still.
15.5.4 Designing the array and the algorithm together⧉
Joint sensor/algorithm design optimizes the measurement pattern together with its reconstruction, subject to a chosen model and objective.
Treat the filter array as parameters, the demosaicker as a network, and train both on the same objective: reconstruct the scene from what this array measures. Chakrabarti (Chakrabarti 2016 (sensor multiplexing)) did this for the multiplexing pattern with backpropagation through a differentiable sampling model; Henz, Gastal and Oliveira (Henz et al. 2018 (joint CFA design)) did it for the array and the demosaicker jointly, and reported reconstruction improvements under their evaluated conditions.
Interpret a learned pattern together with its constraints: permitted filter spectra, periodicity, sensor noise and fabrication limits. A numerical optimum under a simplified model does not establish a manufacturable sensor or a universal advantage over Bayer.
The co-design result is methodological: changing the measurement and reconstruction together can improve the chosen objective. Hardware realization and performance under different spectra, noise and tasks require separate validation.