BookNotesSlidesP-sets Syllabusdraft · v0.2.49.1
◀▲⌂▶
💬Comments welcome. To leave a note, select any text and click the note / highlight button that pops up — or open the panel with the tab at the top-right (‹). Notes are visible only inside our private review group.

4.2 White balance and white point⧉

White balance corrects an image for the scene illumination. A reference white defines a neutral stimulus for a color space or appearance calculation. A display also has a physical white, set by its calibration. These are related quantities, but they are not interchangeable.

For sRGB, equal linear RGB values have D65 chromaticity. ICC’s profile connection space uses a D50 reference. CIELAB accepts a specified reference white; D50 is an ICC convention, not a universal requirement of Lab. The ACES white is near D60. Specifying one of these whites does not itself apply an adaptation transform to an image.

I find the shared terminology unhelpful. A raw file may carry an as-shot white-balance estimate, the working RGB space has a reference white, and the output profile describes another device. Track which operation connects each pair. ACES makes input, working, and output transforms explicit, but it still has a defined working white and output viewing assumptions. It does not remove white adaptation from the rest of the pipeline. See the ACES white-point definition.

The light in the photographed scene and the light around the finished image play different roles. A tungsten-lit room may be shown on a monitor viewed in daylight; the viewer is no longer standing in the captured room. The rendering must decide how much of the scene's warm appearance to preserve under the new viewing conditions. Calling both sides “white” conceals that decision. The raw file's as-shot gains, the working space's reference white, and the calibrated display white are three pieces of information, not three instructions to neutralize the image independently.

This is why double adaptation is such an easy mistake. A conversion may already include the adjustment between its source and destination references. Adding another adjustment because the names D50 and D65 appear in the metadata applies the intended change twice. A pipeline with designated input and output transforms makes that responsibility easier to inspect. That is the useful architectural lesson from ACES; it does not depend on claiming that ACES has no working white or no later viewing adaptation.

4.2.1 Von Kries adaptation and white balance⧉

The von Kries model scales cone responses independently. Writing the three responses as $\mathbf r$, with source and destination white responses $\mathbf w_s,\mathbf w_d$, full adaptation takes the form

$$\tilde{\mathbf r}=D\mathbf r,\qquad D=\operatorname{diag}(\mathbf w_d/\mathbf w_s).$$

Division is componentwise, and the two whites must have a specified common normalization. Camera white balance similarly scales the camera’s linear channels, but these are sensor responses, not cone excitations. A diagonal transform in one basis is generally a full matrix in another. If $\mathbf c=M\mathbf r$, then

$$\tilde{\mathbf c}=M D M^{-1}\mathbf c.$$

In general $MD\ne DM$. A common scalar exposure gain commutes with $M$; three unequal channel gains usually do not.

fig-von-kries
Figure 3.3.16 (reproduced). Von Kries white balance: three independent channel gains. The correction is a diagonal map: scale R, G, and B by separate constants $k_R, k_G, k_B$, chosen so a scene neutral becomes equal in all three channels. Geometrically it stretches the color cube independently along each axis. Simple, cheap, and a surprisingly good model of the eye's own adaptation.
fig-von-kries-lms
Figure 4.2.1. White balance is three numbers, and here they are yours. Each photograph is scene-linear light from one of the book's own raw files, developed for daylight and then relit by a von Kries transform from D65 to the light named beside it, one gain per cone rather than a tint laid over the top: tungsten, a fluorescent, a warm LED, warm daylight, noon daylight, open shade. Your three sliders are those same three gains, so undoing the cast is exactly the inverse, and the figure knows it: the readout says how far you are in stops on the worst cone, and "camera white balance" sets the gains the camera would have used. Or click the picture: an eyedropper sets the three gains so that whatever you clicked comes back neutral, which is exactly what the eyedropper in a raw developer does and exactly the arithmetic of the sliders, the gains being the ratio between a neutral's cone responses and that point's. Click something that is not neutral and it will still be made neutral, which is the failure mode worth having felt. Measured on the open-shade frame: a patch whose channels spread 29 counts apart as shot comes back 1 apart. The neutral wedge under each frame is the other instrument, since eyes are poor judges of an overall cast and excellent judges of whether a gray is gray. Brightness is held fixed, so moving one slider moves the other two to keep a white surface at the luminance it had, which is what a camera does and what makes three gains a two-parameter choice: only the ratios carry the color. Try tungsten and watch how far the S gain has to be pushed, then open shade, where the correction runs the other way and is much gentler. The size of the correction is the distance between the two lights, and the tungsten-to-daylight gap is the widest one photography meets. Temperature and tint are the same correction, named the way a camera names it, and the figure carries both: move them and the three gains follow, move a gain and the temperature and tint that come closest are written back. That is the honest relation between them. The two knobs are a surface inside the three-dimensional space of cone gains, a curve along the Planckian locus and one direction off it, so most gain triples are not a temperature and a tint at all, and when yours is not, the figure says so.

Apply physical channel gains to linear measurements, before clipping or tone rendering. For a pure power encoding $V=L^{1/\gamma}$, the gain $g$ on light corresponds to $g^{1/\gamma}$ on $V$. Applying the same gain to both representations gives different results. Real transfer functions can include offsets and piecewise segments, so decoding is the reliable way to implement a specified linear gain.

fig-wb-linear-vs-nonlinear
Figure 4.2.2. White balance must be done in linear light. The same von Kries gains $k_R, k_G, k_B$ are applied two ways: to linear-light RGB (left), where they cleanly neutralize the cast, and to gamma-encoded RGB (right), where the identical gains additionally shift hue and change apparent saturation because a scaling does not commute with the non-linear encoding. The error is a visible color twist rather than a brightness change, which is why white balance lives early, in the linear pipeline.

Temperature and tint are a convenient interface to illuminant chromaticity. Temperature follows the Planckian locus, obtained by integrating a blackbody spectrum against color-matching functions. Correlated color temperature (CCT) assigns a nearby locus temperature to a source that need not be a blackbody. It does not determine that source’s spectrum.

To construct the temperature axis, start with Planck's law for spectral radiant exitance,

$$M(\lambda,T)=\frac{2\pi h c^2}{\lambda^5}\frac{1}{\exp(hc/(\lambda k_B T))-1}.$$

Here $h$, $c$, and $k_B$ are Planck's constant, the speed of light, and Boltzmann's constant. Use compatible wavelength units, integrate this spectrum against the XYZ matching functions as in Chapter 4.1, Linear color reproduction and CIE XYZ, and normalize to chromaticity. A common multiplier cancels in that normalization. Repeating the calculation over temperatures traces the locus. Temperature supplies one coordinate along the curve; tint supplies a second direction away from it. This explains why two controls can describe a two-dimensional range of white chromaticities even though neither determines the full illuminant spectrum.

The standard signed distance $D_{uv}$ uses the CIE 1960 $uv$ diagram:

$$u=\frac{4X}{X+15Y+3Z},\qquad v=\frac{6Y}{X+15Y+3Z}.$$

Near the locus, a model can specify $(u,v)_{\rm light}=(u,v)_{\rm Planck}(T)+D_{uv}\hat{\mathbf n}(T)$, with a chosen normal direction. Do not use CIE 1976 $v'$ in this formula; it is $3v/2$. Camera “tint” scales are application-specific and need not equal $D_{uv}$. Daylight models also differ from blackbody models. After removing common exposure scale, positive three-channel gains have two degrees of freedom; temperature and tint can parameterize a useful region of them. The mapping depends on the camera calibration.

4.2.2 How do you know the illuminant?⧉

A spectrally neutral, unclipped reference patch measures the illuminant through the camera sensitivities, up to a common scale. For its black-subtracted linear values $(R_n,G_n,B_n)$, gains

$$k_R=g/R_n,\qquad k_G=g/G_n,\qquad k_B=g/B_n$$

map the patch to $(g,g,g)$. Choosing $g=G_n$ leaves the green channel unchanged. Choosing the geometric mean makes the product of gains one; it does not generally preserve luminance. The raw developer’s subsequent camera transform must use the same normalization convention.

Average a patch rather than one noisy pixel. Reject clipped channels and values near the black level, where division is unstable. A wall that looks white may be colored, fluorescent, or lit differently from the subject.

Without a labeled neutral, automatic white balance estimates the illuminant using assumptions about scene statistics or learned scene content. Gray-world and white-patch methods illustrate those assumptions and their failures; the estimators are developed in Chapter 8.10, Auto-exposure and auto white balance.

4.2.3 White balance with mixed illuminant⧉

A face beside a tungsten lamp and a daylight window receives a spatially varying mixture of spectra. One global gain triplet cannot generally neutralize both sources. Local white balance estimates a gain field, but separating material color from illumination requires additional assumptions or user input.

The spectral model multiplies reflectance by the local sum of illuminant spectra before integration. A per-channel product of “material RGB” and “illuminant RGB” is only an approximation to that operation. Chapter 24.2, Mixed-lighting white balance develops methods that work with this ambiguity.

fig-local-wb-cat
Figure 4.2.3. Local white balance as a per-pixel factorization, on real mixed-lighting data. Under a warm lamp on one side and cool daylight on the other, the raw capture is, pixel by pixel, a white-balanced image times a spatially varying color coefficient: $\text{input}=\text{coefficients}\times\text{white-balanced}$. A global white balance would collapse the whole coefficient panel to one constant triplet; the local problem is to recover a different triplet at every pixel, which is why it needs a strong prior on how illumination varies. Data from Boyadzhiev et al., "User-Guided White Balance for Mixed Lighting Conditions" Boyadzhiev et al. 2012.

Consider a gray object between a warm lamp and a daylight window. Correcting the lamp-lit side with one set of gains can make the window-lit side too blue; correcting the window-lit side can leave the other side orange. A spatially varying correction is therefore a different estimation problem from choosing one white-balance triplet. In the approximate product model, each observed pixel combines three unknown material components and three unknown illumination components. Three measured channels cannot identify both triplets independently. A method must couple pixels through assumptions about surfaces, lighting, or user-specified neutrals. That coupling is what makes local correction possible, and also what can make it mistake a colored object for colored light (Figure 4.2.3).

4.2.4 White point in color space definitions⧉

An RGB space specifies primary chromaticities and a reference white. Let the columns of $P$ give primary XYZ directions with unit $Y$. Solve

$$P\mathbf s=\mathbf w,\qquad M_{\mathrm{RGB}\to\mathrm{XYZ}}=P\operatorname{diag}(\mathbf s).$$

Then $M_{\mathrm{RGB}\to\mathrm{XYZ}}(1,1,1)^T=\mathbf w$. This fixes the primary scales and luminance weights. It is a definition of coordinates, not a chromatic-adaptation model. Changing the reference white changes that definition; predicting how a stimulus should look under another adapting white requires an additional model.

Viewing adaptation may be partial and can involve both the image and its surround. A monitor’s brightest patch is not guaranteed to appear neutral regardless of its chromaticity. Color-management workflows specify reference viewing conditions so these assumptions can be shared.

The three roles should be named separately: a coordinate normalization, a physical display calibration target, and an assumption about the viewer's adaptation. They are often chosen to work together, but none logically forces the other two. The matrix equation above selects how much each primary contributes to the reference neutral. It does not establish what a viewer in an arbitrary room will perceive as white. My objection to the terminology is that a rendering assumption can become hard to see once it travels under the same “white point” label as the coordinate definition. Keep both in the specification rather than inferring viewing conditions from an RGB triplet.

4.2.5 Spectral sharpening within LMS space⧉

A chromatic-adaptation transform predicts corresponding colors across adapting conditions. A common linear Bradford implementation changes basis, scales the white responses, and changes back:

$$M_A=\begin{pmatrix}0.8951&0.2664&-0.1614\\-0.7502&1.7135&0.0367\\0.0389&-0.0685&1.0296\end{pmatrix},$$
$$\boldsymbol\rho_s=M_A\mathbf w_s,\quad \boldsymbol\rho_d=M_A\mathbf w_d,\quad \mathbf c_d=M_A^{-1}\operatorname{diag}(\boldsymbol\rho_d/\boldsymbol\rho_s)M_A\mathbf c_s.$$

Here the whites and colors are XYZ. $M_A$ alone is the basis transform, not the complete white adaptation. For full adaptation with normalized whites, the complete transform maps $\mathbf w_s$ to $\mathbf w_d$. Source and destination white equal gives the identity, useful checks on an implementation.

rho_s = M_A @ source_white_XYZ
rho_d = M_A @ destination_white_XYZ
require every rho_s component is safely nonzero
adapted = solve(M_A, (rho_d / rho_s) * (M_A @ color_XYZ))

Bradford’s effective response functions are linear combinations of the XYZ matching functions Bradford chromatic-adaptation transform. They can have signed lobes. These are numerical coordinates, not new physiological cone measurements.

fig-bradford-responses
Figure 4.2.4. What the Bradford matrix is, drawn as three curves. Bradford is usually met as nine numbers; it is easier to believe once you see what they do. Each row of the matrix, dotted with the color-matching functions, is a spectral sensitivity, and the three of them are recognizably the cones, only narrower. Both sets shown are rows of a $3\times3$ acting on XYZ, so the comparison is like for like: the Hunt-Pointer-Estévez rows, the von Kries cones the classical model adapts in, against the Bradford rows fitted to corresponding-colors data. Two properties are measured on the curves rather than asserted. The equivalent bandwidth, the width of the rectangle with the same area and the same peak, falls from 111 and 99 nm for the long and middle channels to 95 and 85 nm. The overlap between those two, the pair that overlaps badly and the pair sharpening is really about, falls from 0.93 to 0.62. And the Bradford curves carry small negative lobes, dipping to about a tenth of their peak, which broad non-negative cone responses cannot do.

CAT16 supplies a different response basis in the CAM16 appearance model Li et al., CAM16 and CAT16 (2017):

$$M_{\mathrm{CAT16}}=\begin{pmatrix}0.401288&0.650173&-0.051461\\-0.250268&1.204414&0.045854\\-0.002079&0.048952&0.953127\end{pmatrix}.$$

The full appearance model includes adaptation degree and nonlinear response processing. Replacing the Bradford basis above with this matrix gives a full-adaptation transform, not all of CAM16. Use the transform required by the workflow: substituting CAT16 inside a profile conversion that specifies Bradford changes the conversion.

Corresponding-color experiments ask observers to match appearance under different conditions. Spectral relighting instead computes how the same reflectance responds to two illuminants. The figures below test the latter objective. Their rankings cannot establish which full appearance model best predicts people, and full adaptation is a valid limiting case rather than an inherently invalid test setting.

The architecture of the appearance model explains another difference between CAT02 and CAT16. CIECAM02 used one response space for chromatic adaptation and then changed to a Hunt–Pointer–Estévez response space for later processing. CAM16 instead performs the adaptation stages within one response space. Li and colleagues introduced this restructuring to address numerical and range problems while maintaining the model's fit to appearance data Li et al., CAM16 and CAT16 (2017). Thus the choice is about the surrounding model as well as the entries of an adaptation matrix. A comparison that substitutes only the three-by-three matrix does not test the whole architectural change.

Corresponding-color data and spectral relighting also answer different questions. In a corresponding-color experiment, a person chooses the patch that looks like a match under another adapting condition. That judgment depends on the surround, light level, and the observer. A spectral test instead measures or predicts what a fixed surface emits under the other illuminant. An appearance model can deliberately retain a cast when adaptation is incomplete, even though a physical relighting comparison rewards complete neutralization. Before reading a ranking of transforms, identify which target and which viewing assumptions generated the errors.

4.2.6 The limits of white balance, and CRI (color rendering index)⧉

Two illuminants can have the same white chromaticity yet render colored objects differently. If a source delivers little or no energy in bands that distinguish two reflectances, three channel gains cannot recover the missing distinction. Even smooth illuminants need not be related by an exact diagonal correction.

The color rendering index (CRI) compares specified test colors with a reference source at a similar correlated color temperature. The commonly quoted general index $R_a$ averages eight test colors; it is not a complete assessment of saturated colors, preference, or camera fidelity. A high score does not specify the illuminant spectrum. Contemporary lighting specifications may also report more detailed fidelity and gamut measures. For photographic work, examine the actual source and the materials that matter, especially when mixing lights.

fig-cri-comparison
Figure 4.2.5. High versus low color rendering index. A ColorChecker chart rendered under a high-CRI reference (D65, CRI 100) and under a low-CRI spiky fluorescent (the CIE standard "F-series" fluorescent illuminant number four, CRI $\approx 52$), each white-adapted so the neutrals stay neutral, so the chromatic distortion is what shows. Both spectral power distributions and CRI scores are printed; the patches the spiky spectrum mis-renders are outlined. White balance keeps the grays true but cannot recover the colors the gappy spectrum never lit.
fig-white-balance
Figure 4.2.6. The limits of white balance, interactively. A Macbeth ColorChecker is rendered from spectral reflectances under a chosen illuminant (daylight D65, tungsten A, a spiky fluorescent, or a warm-white LED; the fluorescent is a synthetic spectrum with peaks at 430, 490 and 570 nm, not CIE F2, and the camera channels are XYZ converted to linear sRGB, not measured sensor sensitivities). The demo shows the captured chart beside the D65 ground truth, the selected patch's resulting spectrum, and the best correction by a diagonal (von Kries) gain versus a full $3\times3$ matrix, each fitted by least squares in linear RGB, with the mean CIE76 $\Delta E$ printed after gamut clipping, plus manual temperature/tint sliders. Smooth illuminants usually correct well, though not always exactly; spiky ones tend to leave a residual that no $3\times3$ removes. A nonzero residual alone does not identify a particular metameric pair; it shows that three gains or nine coefficients cannot undo a change that happened wavelength by wavelength.

A matrix fitted on chart patches can reduce their residuals without accurately correcting other materials. Separate fitting and evaluation sets when testing a camera transform. Neither a low training error nor a neutral gray row establishes general spectral recovery.

4.2.7 Spectral sharpening with general spectra⧉

Spectral sharpening recombines measured channels to make their effective sensitivities more localized Finlayson, Drew & Funt, spectral sharpening (1994) Süsstrunk, Holm & Finlayson, chromatic adaptation sharpening (2000) Barnard, Ciurea & Funt, sensor sharpening for color constancy (2001). It can be applied to camera sensitivities as well as cone-based coordinates. A numerical three-by-three transform remains within the span of the original three sensitivities; it cannot synthesize arbitrary new spectral measurements.

For reflectance $r(\lambda)$, illumination $e(\lambda)$, and sensitivity $s_i(\lambda)$, a channel measures

$$c_i=\int r(\lambda)e(\lambda)s_i(\lambda)\,d\lambda.$$

A gain accurately models a change of illumination when the ratio of the two illuminant spectra is nearly constant across that channel’s effective support. Infinitely narrow sensitivities are one sufficient idealization; a common scalar change of illumination is another. Narrowing the effective support may help, but sharper is not universally better.

Numerical recombination also changes noise. For transform $H$ and input covariance $\Sigma$, the output covariance is

$$\Sigma_{\rm out}=H\Sigma H^T.$$

Large signed combinations can amplify noise and create channel correlations. A condition number alone does not give output noise variance. Nor does an invertible change of coordinates by itself destroy information; the practical costs arise in the operations, quantization, or clipping performed in those coordinates. Physically narrowing absorbing filters is different: it reduces detected photons under otherwise comparable conditions. The sensor noise model is developed in Chapter 2.8, Noise, signal-to-noise ratio and dynamic range.

fig-spectral-sharpening
Figure 4.2.7. How well does white balance actually work? (interactive). White balance is one gain per channel, applied in some basis, that undoes an illuminant. It can only ever be an approximation, because the illuminant did not scale three channels, it multiplied the spectrum wavelength by wavelength. So the figure runs the experiment instead of asserting the result: for every patch, take its real tristimulus under the scene light, apply the diagonal correction back to daylight, and compare against the truth, the same surface under daylight. The three rows are the same patches as shot, as corrected, and as they really are; the bars are what is left over, in $\Delta E_{00}$, on an axis that is fixed across every setting so that changing the light or the basis moves the bars rather than the scale. It uses the same 24 ColorChecker patches as Figure 4.2.6. The basis matters, in the order the theory predicts: the cones are the classical choice, and the sharpened bases, which recombine them into narrower, less overlapping channels, do better. Under tungsten the cones leave 4.10, Bradford 2.04, CAT16 3.21, and a basis derived from the sensors alone 1.91. That last one is fitted to nothing: each of its channels is the combination of the three color-matching functions that puts as much of its energy as possible inside one narrow band, which is spectral sharpening in its original form, and it matches a Bradford matrix that was fitted to corresponding-colors data. CAT16 trailing Bradford is not an artifact of the test: it is the newer transform, but it is also the less sharp of the two, and here the sharper basis wins. Sharpening is the mechanism, not the badge. The last two bases are what software actually reaches for, and only one of them is a mistake. Applying the gains to the display's own linear channels leaves 3.62: worse than either sharpened basis, and on this chart better than the cones, which is worth sitting with. The sRGB primaries were chosen for a display, not for adaptation, and they still beat the basis the eye is believed to adapt in, which is a large part of why cameras scaling their own raw channels has always worked well enough. Applying the gains to the gamma-encoded numbers, which is what a white-balance slider on a JPEG does, leaves 5.11 and is the worst of the six. The white patch still comes out neutral, since the gains are set by the white, but the gray wedge tilts. The reason is narrower than "do it in linear light." A per-channel gain and a pure power law commute: scaling an encoded value by $g$ and decoding is exactly scaling the light by $g^{2.4}$, so with a pure 2.4 power the two agree to four decimals and give the same 3.62. What breaks it in sRGB is not the power but the toe and the offset, the linear segment near black and the 1.055 and 0.055 in the curve, which turn a scaling into a scaling plus a black shift. The reasoning does not generalize to averaging: a mix of two colors, unlike a scaling of one, does not commute with the encoding at all. The spectrum matters, not just the color of the light: choose fluorescent F11, then F11's smooth twin, a blackbody warped until its tristimulus matches F11's exactly. The white point and the correction it implies are then identical, and the only difference is that one light is made of mercury lines. In the Bradford basis the mean error goes from 1.02 to 2.11 and the worst from 2.32 to 6.48. A camera that estimates a white point rather than a spectrum has a floor it cannot get under, and that floor is the real limit of white balance. And the patches matter: the six neutrals come out near zero by construction, since the gains are set by the white, so every chromatic patch is where the model has to do real work, and the residue grows with how far the surface sits from neutral. Click any patch to see its reflectance and what each light makes of it.
fig-wb-method-grid
Figure 4.2.8. Every patch against every method, twice (interactive). The same experiment as Figure 4.2.7 and the same 24 ColorChecker patches, transposed: a column per surface, a row per treatment, the scene as shot at the top and the truth, those surfaces in daylight, at the bottom. Read a column downwards and you are watching seven treatments of one surface; read the bottom row against the top and you see what the light did before anything corrected it. The lower grid is the same table in $\Delta E_{00}$, which is the part the eye cannot do by inspection: tell a 1.8 from a 3.2. Its ramp is fixed under every light and deliberately not linear, with stops at 0, 1, 2.3, 5, 10, 20 and 30 spaced evenly, because a linear scale to 30 would paint every basis the same pale yellow. Two columns stand to the right: each row's mean under the light on screen, and its mean over every light in the picker, which is the column to read if the question is which basis to choose rather than how this one behaves under tungsten. Over all six lights: Bradford 1.21, the sensor-derived sharpening 1.27, CAT16 1.67, linear sRGB 2.02, the cones 2.20, the gamma-encoded gains 2.54. Three things are visible here that the other layout hides. The six neutral columns are flat for every basis except the last: the gains are set by the white, so the grays come back for free, and the one row where they do not is sRGB, gamma-encoded, the only treatment that is not a gain on light. That row is the case for balancing in linear light, as a picture rather than a claim. The error collects on the same columns for every basis, the saturated ones: cyan, orange, magenta. A per-channel gain is worst where the reflectance is most peaked, and the choice of basis moves how much is left, not where. And the as-shot row sets the scale: under tungsten it averages 18.7 and runs to 28.7, so a basis that leaves 2 has done almost all the work, and the argument between the bases is about the last few percent. Point at a column to name the patch and read its numbers.
fig-hyperspectral-relight
Figure 4.2.9. White balance on a whole scene, in the comparison viewer (interactive). Every pixel carries a full reflectance spectrum, 29 samples from 420 to 700 nm, so the scene can be relit: rendering it under any illuminant is an integral of reflectance times that illuminant against the color-matching functions, computed for all 65,536 pixels. Four panes over one scene, and each pane's buttons choose what it shows: the scene as shot, white balanced in any of six ways, the ground truth, or the error of any of them against it, whether or not that one is on screen. Drag the splits to compare, scroll to zoom, or drop to one or two panes. The reference is not another algorithm's output but what those surfaces really look like in daylight, computed from their spectra, so the PSNR beside each pane means what it says. Three of the bases are the usual suspects: the cones, and the two published sharpenings, Bradford and CAT16. The fourth is derived rather than published, and derived from the sensors alone: each channel is chosen to put as much of its energy as possible inside one narrow band, which is spectral sharpening in its original, sensor-based form. It is one matrix, good for every light, fitted to no illuminant and no perceptual data. On the chart scene under tungsten the four give 38.6, 41.7, 37.1 and 42.5 dB, so a basis that knows nothing but the color-matching functions edges out the one fitted to corresponding-color data. The last two are what software does rather than what color science recommends: the same gains applied to the display's linear channels, 31.2 dB, and applied to the gamma-encoded values, 27.5 dB, the case that leaves the grays tilted because a gain on an encoded number is not a gain on the light. The band width is where the interesting part lives, because sharpening is not monotone. Sweeping it and measuring the adaptation error on the ColorChecker: at $\pm20$ nm the channels are nearly orthogonal and the error is 3.61 $\Delta E_{00}$; at $\pm100$ nm they are broad again and it is 3.30; at $\pm50$ nm it bottoms out at 1.96, against Bradford's 2.02 and the raw cones' 4.10. Sharpening helps up to a point and then hurts, which is exactly the bounded gain this section describes. The error view is the thing to look at, and it can be read in $\Delta E_{00}$ rather than as an amplified difference. It is not flat: white balance is near-exact on neutrals, because the gains are fixed by the white, and it fails on saturated, spectrally peaky materials. On the stuffed-toy scene under tungsten the residue collects on the yellow and red stripes of the sweater and on the red nose while the gray chart beside them stays at zero. The error map is a map of materials. The CAVE scenes are measured reflectance, so the ground truth is a measurement; the Harvard ones are radiance under daylight, so their reflectance rests on an assumed daylight spectrum.

For a simple example of the noise cost, subtract two channels that both respond strongly to the same broad spectral component. Their shared signal can nearly cancel while their independent read noise does not. If each channel has noise standard deviation one and the noises are independent, their difference has variance two and standard deviation about 1.41. Multiplying that small difference to make it a useful color coordinate multiplies the noise as well. Correlated channel errors change the calculation, which is why the covariance matrix matters. This is a numerical recombination of already captured photons. Placing a narrower absorbing filter in front of the sensor instead changes which photons are detected in the first place.

To see why localization helps, consider a channel that responds across both sides of a narrow peak in the illuminant. Two materials can put different proportions of their reflected energy under that peak while producing the same channel value under the original light. Under the new light they change by different factors. One gain cannot represent both changes. A more localized effective sensitivity samples a smaller range of those factors, which can make a diagonal approximation more accurate. Numerical sharpening seeks such useful combinations within the measurements already available; it cannot reveal spectral differences that all three original channels failed to distinguish.

4.2.8 Subjective white balance⧉

A neutralized gray card is useful for calibration, but it need not be the desired photograph. Candlelight and sunset illumination contribute to a scene’s appearance. A rendering may retain some warmth instead of fully discounting the light. Human adaptation itself is often incomplete.

Keep illuminant estimation, appearance prediction, and artistic preference separate. A dataset with measured neutral targets evaluates estimation. Corresponding-color experiments evaluate appearance matches. Expert retouches, such as MIT-Adobe FiveK Bychkovsky et al. 2011, provide examples of preferred rendering, though their edits include much more than white balance. Neither a person’s choice nor a manufacturer’s rendering is a unique physical ground truth.

Skin, sky, foliage, and the mood of the illumination can influence rendering decisions; see Chapter 4.5, Advanced color: opponents, perceptually-uniform, and preferred. Automatic systems can use scene content in both estimation and rendering, but a good result should not be interpreted as evidence that the illuminant was recovered exactly.

A candlelit dinner makes the distinction visible. A gray-card correction can provide a neutral reference, but applying it fully may remove the warm appearance the photographer wants to retain. Compare that neutral version with a partial correction: the latter leaves some cast while still separating skin from the orange illumination. Neither comparison requires treating one rendering as the true reflectance of every surface. The neutral reference is useful for calibration; the residual warmth is a rendering choice. A preferred white balance must therefore be evaluated with people and scenes, not only with a neutral-patch error.

This distinction changes how we diagnose a camera's output. Suppose a neutral target gives a reliable illuminant estimate, but the camera renders a tungsten-lit room warmer than a gray-card correction. The residual cast can be a deliberate choice about the photograph. Re-estimating the illuminant will not explain that choice; we need to inspect how the estimate is converted into rendering gains. Conversely, a camera can make an attractive image while estimating the illuminant poorly. A pleasing result by itself does not establish colorimetric accuracy.

A useful conceptual pipeline estimates the light first and chooses how much of its appearance to preserve second. It can fully neutralize a product photograph while leaving some warmth in a candlelit portrait. These choices need not be exposed as two separate controls, or even implemented as separate modules, but their objectives differ. The illuminant estimate can be checked against a reference target; the preferred warmth needs judgments about the rendered scene.

Scene recognition can enter both decisions. Recognizing a face may supply evidence about illumination, although skin reflectance varies and is not a gray card. It may also influence the desired rendering of skin after the light has been estimated. Sky and foliage raise similar distinctions between the measured color, the color a viewer remembers, and the color the photographer wants. A global white-balance gain changes the whole picture: independently adjusting skin, sky, and foliage requires additional selective color processing, as discussed in Chapter 4.5, Advanced color: opponents, perceptually-uniform, and preferred.

Manufacturer looks and user preferences add another choice. A rendering style can favor a warmer portrait or a cooler overall balance while starting from the same estimate of the light. Such choices are part of what photographers call a camera's color science, alongside its tone curves and selective color adjustments. They can differ between products and audiences; there is no single preferred white point that physics determines for every photograph.

The two kinds of training target therefore deserve separate treatment. An illuminant dataset records a physical reference and asks an estimator to recover it without seeing the chart. Expert-retouched photographs record choices about a finished image. The five retouchers in MIT-Adobe FiveK supply different renditions of the same inputs, which is useful precisely because preference need not have one answer. Their edits include exposure and tone as well as color, so using them to learn white balance requires distinguishing those operations. Scoring a preferred rendering solely by its distance from a neutral target would penalize some of the decisions it was designed to make.

An appearance model occupies yet another position. Its corresponding-color data asks what looks the same under different viewing conditions. That is an empirical model of human response, including incomplete adaptation. Preference asks what looks good. A mismatch to an appearance experiment calls for checking the viewing assumptions and model; a disagreement over the warmth of a photograph can remain after those are correct.

4.2.9 White balance datasets⧉

Public color-constancy datasets provide reference illuminants for training and evaluation. The common recipe runs the interactive figures above in reverse: photograph a real scene with a known neutral target in the frame (a gray card, a Macbeth ColorChecker, or a gray ball or cube), read the illuminant color off that target, then mask the target out so the algorithm has to recover the same illuminant from the scene alone. Accuracy is often reported as the angular error between the estimated and the true illuminant chromaticity, quoted as mean, median, and the worst 25%, since the hard scenes are the ones that matter. Two cautions run through all of them. The ground truth is only as trustworthy as the target and its placement: one card cannot describe mixed light. And the encoding of the released images matters enormously, black-level subtraction and whether the pixels are linear raw or already gamma-tagged shift the reported numbers, which is the well-documented source of the ColorChecker "reprocessing" discrepancies (Banić et al. on the ColorChecker dataset misuse).

Single-illuminant benchmarks, one global light per image, are the mainstay:

Mixed and multi-illuminant datasets are the harder, more realistic frontier, exactly the regime where a single global gain cannot win:

For post-capture correction, fixing the white balance of an already-rendered sRGB image rather than estimating a raw illuminant, the reference set is Rendered WB / When Color Constancy Goes Wrong (Afifi et al.): roughly 65,000 sRGB images rendered under many camera white-balance settings, for learning to re-neutralize a mis-balanced JPEG.

Finally, the spectral side of the problem needs images that carry the full per-pixel spectrum together with a known or controlled illuminant, so you can compute a reference illuminant chromaticity and predict the scene under another light within the measured spectral range, studying the metameric failures of the previous section directly rather than through one camera's fixed sensitivities: