15.3 Mosaics and aliasing⧉
A color filter array samples each color channel on a lattice sparser than the pixel grid: in a Bayer mosaic, red and blue are measured at one photosite in four and green at one in two. Chapter 6.4, Sampling, downsampling, and aliasing tells us what a lattice can hold. Detail finer than the lattice supports does not disappear; it reappears at a lower frequency, as an alias. On a mosaic the alias has a color, which is why fine gray texture turns into false color and colored moiré. This chapter looks at the mosaics in the Fourier domain, one channel at a time and then all channels together, for Bayer, X-Trans, Quad-Bayer and Nona-Bayer. It then turns to the optics that try to remove the offending detail before it is sampled: the anti-aliasing filter, which is disappearing from cameras, and the nano-prism color router, which does part of its job without being designed for it.
Throughout, frequencies are in cycles per pixel, so the pixel grid itself can hold everything up to $1/2$ in each direction.
15.3.1 Each channel lives on a sparser lattice⧉
Sampling one channel of a mosaic is a multiplication: the full-resolution channel $x_c$ times a mask $m_c$ that is 1 where the array has color $c$ and 0 elsewhere. A product in space is a convolution in frequency, so the spectrum of the sampled channel is the spectrum of $x_c$ convolved with the spectrum of the mask. A periodic mask has a spectrum made of impulses, one at zero frequency, with height the fraction of sites of that color, and others at frequencies set by the period of the array. Each impulse places a copy of the channel's spectrum at its position. The copies away from zero are the aliases, and a channel is recorded without aliasing only if its spectrum is narrow enough that the copies do not overlap the original.
For Bayer red, the mask repeats every two pixels in each direction. Its impulses sit at zero, at $(1/2, 0)$, $(0, 1/2)$ and $(1/2, 1/2)$, each of height $1/4$, so red is alias-free only if it holds nothing above $1/4$ cycle per pixel, horizontally or vertically: half of what the pixel grid supports. Blue is the same lattice shifted by one pixel diagonally, which changes the signs of its impulses and not their positions. Green occupies a checkerboard, the quincunx lattice, whose mask has a single impulse away from zero, at the diagonal frequency $(1/2, 1/2)$. Its alias-free region is a diamond, $|f_x| + |f_y| < 1/2$: green can hold the full $1/2$ cycle per pixel along the axes, like the pixel grid, but only $1/4$ along the diagonals, like red. Along the diagonals, then, green holds only half of what the pixel grid can, one frequency view of the maze and zipper patterns of Chapter 15.1, Basic traditional demosaicking.
Figure 15.3.1 draws these impulses for four arrays, with each channel's alias-free region, and the spectrum of a real mosaic channel by channel.
15.3.2 All channels at once: luma and chroma⧉
The mosaic is one image, not three, and its spectrum is the sum of the three sampled channels. Because the masks add up to one at every pixel, their impulses away from zero cancel for a gray scene: whatever sits at a carrier frequency is a difference between channels. For Bayer, with red at even-even positions and blue at odd-odd positions, the algebra is exact and short:
At even-even positions the expression gives $R$, at mixed parity $G$, and at odd-odd positions $B$, which is a direct implementation check; a different origin for the array changes the signs of the carriers. The luma $L$ is a mixture of camera channels, not CIE luminance. Multiplication by $(-1)^i$ shifts a spectrum by $1/2$ cycle per pixel, so the mosaic is the luma at baseband plus $C_1$ modulated onto the two axis carriers $(1/2, 0)$ and $(0, 1/2)$, plus $C_2$ modulated onto the diagonal carrier $(1/2, 1/2)$ (Alleysson, Süsstrunk & Hérault 2005 (frequency-domain demosaicing); Dubois 2005 (frequency-domain demosaicking)). Given separated components, $G=L-C_2$, $R=L+C_2+2C_1$ and $B=L+C_2-2C_1$.
The luma is sampled at every pixel and keeps the full band of the pixel grid. The chroma is what pays for the mosaic: it is recorded only as copies parked at the highest frequencies the grid supports. This works because natural images have much less chroma detail than luma detail, so the chroma copies are compact and mostly stay clear of the luma band. Chapter 15.2, Advanced traditional demosaicking used this model as an algorithm, filtering the chroma copies out and shifting them back; here it is a way of seeing what goes wrong.
15.3.3 False color and moiré as spectral overlap⧉
Aliasing on a mosaic is the luma band reaching a chroma carrier. Fine gray detail near $1/2$ cycle per pixel, a fence, a woven fabric, the fine print on a page, puts luma energy exactly where a demosaicker looks for chroma. The demosaicker cannot tell the two apart and reports a color the scene did not have: false color. A pattern whose frequency is close to a carrier, but not on it, beats against the carrier and produces a slow colored ripple: color moiré. Where the luma band and a chroma copy overlap, the mosaic does not determine which is which, and no reconstruction can separate them without an assumption about the scene. Image context and priors, including learned ones (Chapter 15.6, Learned demosaicking), make better guesses, and extra captures (Chapter 15.12, Hardware alternatives to the mosaic) remove the ambiguity; none of this recovers arbitrary overlapping signals from one mosaic.
Foreshortening makes this common. A fine pattern on a surface that recedes from the camera, a floor, a roof, a striped shirt turned away, grows finer on the sensor with distance, so it sweeps through every frequency on its way to the horizon and passes the carriers at some distance or angle, where the colors appear (Figure 15.3.3).
The same picture explains the two greens. Green carries half the luma, so doubling it keeps the luma band wide, while red and blue enter mostly through the chroma, whose narrow band the sparser lattice can hold. This is a statement about natural images and about the eye, which also resolves luma more finely than chroma, and not about green light as such. A sharp colored pattern, red text on a blue sign, violates the assumption and aliases in the chroma itself.
The model is global and images are not. A photograph is smooth in some regions and finely textured in others, so a filter that separates the bands correctly in one place is wrong in another; the adaptive methods of Chapter 15.2, Advanced traditional demosaicking estimate the local structure and change the filter accordingly. The carrier positions describe an array, not the error of a particular demosaicker on a particular image Hirakawa & Wolfe 2008 (spatio-spectral sampling).
15.3.4 X-Trans, Quad-Bayer and Nona-Bayer in the Fourier domain⧉
Any periodic array has the same description, with more carriers when its tile is larger. What changes from one array to the next is where the carriers sit and how strong each one is, which decides which scene frequencies turn into color.
Quad-Bayer repeats every four pixels, so its carriers are at $1/4$ cycle per pixel, $(1/4, 0)$, $(0, 1/4)$ and $(1/4, 1/4)$, and it has none at $1/2$: the $2 \times 2$ blocks of one color act like a box filter that cancels the Bayer carriers. Nona-Bayer repeats every six pixels and puts its strongest carriers at $1/6$ cycle per pixel, with weaker ones at $1/2$. At full resolution, gray detail with a period of four pixels on a Quad sensor, or six on a Nona sensor, already lands on a chroma carrier, where a Bayer sensor of the same pixel count reaches that point only at a period of two pixels. Detail between those frequencies and $1/2$ is still recorded, mostly in the luma, which is why a remosaicking step can recover much of it; but fine gray texture is ambiguous with color at frequencies two or three times lower than on Bayer. In binned mode the question disappears: summing each block produces an ordinary Bayer mosaic at twice or three times the pitch (Chapter 15.5, Alternative color filter arrays), with the block as a pixel aperture that attenuates a little of what would alias.
X-Trans repeats every six pixels, but its layout is designed so that every row and column contains all three colors. Its masks have no impulse at $(1/2, 0)$, $(0, 1/2)$ or $(1/2, 1/2)$, the three Bayer carriers. Its chroma is spread over more carriers, the strongest at $(1/3, 1/3)$ and at $(1/2, 1/6)$ and its symmetric positions, with weaker ones at $(1/3, 0)$ and $(0, 1/3)$. Each carries less energy than a Bayer carrier, and none sits at the highest frequency along the axes, where the regular patterns of fabrics, grids and facades are often aligned. A given pattern therefore produces weaker false color, spread over more colors, which is the case Fujifilm makes for omitting the anti-aliasing filter. The price is that its lowest carriers are at $1/3$ cycle per pixel, below Bayer's, and that its irregular neighborhoods need a demosaicker of their own.
15.3.5 Prefiltering a mosaic: one blur for three lattices⧉
The remedy for aliasing in Chapter 6.4, Sampling, downsampling, and aliasing is to remove the frequencies a lattice cannot hold before sampling, with a low-pass prefilter. A Bayer mosaic asks for three different ones. Red and blue should be cut at $1/4$ cycle per pixel in each direction, green at the diamond $|f_x| + |f_y| < 1/2$, and the luma, which is sampled everywhere, should keep everything up to $1/2$. A camera can only blur light optically, before it reaches the sensor, and an optical blur has two limitations that a digital filter does not have.
First, it is the same for every color, to a first approximation, because the lens and the filter do not know which photosite a ray will land on. A blur strong enough to protect red and blue removes half the resolution of the luma, which the mosaic records perfectly well; a blur gentle enough to keep the luma leaves red and blue to alias. Second, an optical point-spread function is a distribution of light, positive everywhere, so its frequency response cannot have the sharp cutoff of an ideal low-pass filter. It falls gradually, attenuating detail the sensor could have kept and passing some it cannot.
The anti-aliasing filter of most cameras, also called the optical low-pass filter, is a pair of thin birefringent plates, usually crystal quartz or lithium niobate. Each plate splits a ray into two copies displaced by a distance $d$, one plate horizontally and one vertically, so a point of light lands as four spots at the corners of a $d \times d$ square. Its frequency response is the product of two cosines,
which is zero at $f_x = 1/(2d)$ and at $f_y = 1/(2d)$. With $d$ equal to one pixel, the zeros fall exactly on the Bayer carriers, $(1/2, 0)$, $(0, 1/2)$ and $(1/2, 1/2)$. The filter is well aimed: it removes the gray detail that would land on a chroma carrier and become false color, the most common failure. It does little for red and blue themselves. At $1/4$ cycle per pixel, where they should already be cut, it still passes $\cos(\pi/4) \approx 0.71$ in each direction, so fine colored detail aliases within the red and blue lattices and is left to the demosaicker. Doubling $d$ would move the zeros to $1/4$ and protect red and blue, at the cost of blurring the luma to half resolution. Every camera with such a filter chooses a point between the two, and a weaker filter trades less blur for more moiré (Figure 15.3.4).
Greivenkamp proposed a way out: a prefilter whose blur depends on the color, obtained by placing a polarization retarder between the two quartz plates, so that the split is larger for red and blue than for green Greivenkamp 1990 (color-dependent optical prefilter). It addresses exactly the mismatch above, at the cost of a thicker filter stack. Two other blurs act before sampling whether or not the camera has a filter. The photosite integrates light over its own area, a box one pixel wide whose response only reaches zero at one cycle per pixel, so it barely attenuates the Bayer carriers. And the lens blurs, by aberrations and by diffraction, whose blur grows with wavelength: it attenuates red more than green, which helps red, and blue less, which does not.
15.3.6 Anti-aliasing filters are disappearing⧉
For two decades almost every digital camera with interchangeable lenses carried an anti-aliasing filter. Since about 2012 most high-resolution cameras have dropped it or made it optional. Nikon sold the D800E in 2012 with a second birefringent layer that undid the first, and later models omitted the filter entirely; Fujifilm's X-Trans cameras never had one, which was the point of their layout; Sony's A7R and Canon's 5DS R followed. Pentax built a switchable filter out of the stabilization system: in the K-3 of 2013, the sensor traces a small circle during the exposure, which blurs each point over about a pixel, and the photographer turns it on for fabrics and off for landscapes. Phones generally have none, because at their pixel sizes the lens's own blur already attenuates detail near the sampling limit.
Three things made the filter less necessary. Pixel counts rose until many lenses, apertures and shooting conditions no longer deliver detail at the pixel grid's limit, so the lens acts as the prefilter. Demosaickers improved: the edge-directed methods of Chapter 15.1, Basic traditional demosaicking and the residual methods of Chapter 15.2, Advanced traditional demosaicking, and then the learned ones of Chapter 15.6, Learned demosaicking, reconstruct aliased luma detail with much less false color than bilinear interpolation, and raw converters added tools that remove moiré after the fact. And the filter's cost, a slight softening of every photograph, is paid on every image, while its benefit appears only on the few with fine periodic patterns. Moiré has not gone away: fashion and product photographers still meet it on fabrics, which is why the switchable filter exists. But the decision has moved from a fixed piece of glass to the demosaicker and to the photographer.
15.3.7 Nano-prisms as anti-aliasing filters⧉
A color router, or nano-prism, replaces the microlens above each photosite with a nanostructured layer that steers each band of light toward the photosite whose filter passes it, instead of letting that filter absorb it (Chapter 4.6, Sensing color: multiplexing strategies, Chapter 15.12, Hardware alternatives to the mosaic). Samsung's first sensor with one, the ISOCELL JNP, went into production in 2025 and keeps its color filters under the router; Samsung reports about 25% more light reaching the photodiodes (Samsung). The router is sold as a light-efficiency device, but it also changes what each photosite integrates, and that makes it an anti-aliasing filter.
In the ideal limit, a Bayer router sends all the red light falling on a $2 \times 2$ tile to the tile's red photosite, and all its blue light to the blue one, while the green light falling on each red or blue site is shared among its four green neighbors, so each green photosite collects green from its own site and a quarter of each of its four red and blue neighbors. Red and blue are then integrated over a box two pixels wide rather than one. That is a prefilter matched to their lattice: its response is zero at $1/2$ cycle per pixel and down to about $0.64$ at $1/4$, where red and blue alias, while green, whose lattice is denser, is integrated over half that area. Along an axis the difference is small at $1/4$ cycle per pixel, where green keeps 0.68 against red's 0.64, because green's footprint also spreads sideways; it is clear near $1/2$, where green keeps about a third and red and blue nothing. It is the color-dependent prefilter that Greivenkamp built from quartz and a retarder, obtained here as a side effect of collecting more light. Its attenuation is modest, far from an ideal cut, but it is applied to the channels that need it and spares the one that does not (Figure 15.3.5).
Real routers are not ideal. They send part of each band to photosites of the wrong color, which mixes the channels' spectral responses and has to be undone by a stronger color-correction matrix, and their collection footprint depends on wavelength and on the angle of the incoming light, so it varies across the frame. What a demosaicker should assume about a router sensor is therefore a measured footprint per channel, not the ideal box. The direction is still clear: routing makes each sparse channel collect light from an area closer to its own sampling pitch, which is what an anti-aliasing filter for a mosaic should do, and it does so without discarding light.
Figure 15.3.6 puts the four arrays through the same test, a gray zone plate and a woven texture.
