8.2 What the numbers mean🔗⧉

Images usually encode a pixel as a triplet of numbers, but the exact meaning of those numbers can vary significantly. The numbers usually stand for photometric quantities (think of the number of photons received at a photosite), but they may be encoded non-linearly. And different images may use different color spaces, and even RGB comes in many variants. A large share of image-processing bugs ("why are my colors wrong, my blend dark, my HDR muddy?") trace straight back to encoding and color-space mismatches.

8.2.1 What the numbers actually mean: encoding🔗⧉

The same stored value can represent different light levels under different transfer functions. This matters immediately when averaging two images or adjusting exposure. Use the space required by the operation, and convert deliberately:

• Linear values can be proportional to scene light or to display light. Physical light mixing uses linear values in the relevant domain. A model-aware denoiser or appearance filter may use another representation.
• Gamma-like encodings allocate more code values to dark linear-light intervals. They are useful for storage and display interfaces, but are not exactly perceptually uniform.
• Log encoding represents multiplicative exposure changes as additive offsets. Scaling log contrast can compress a range; subtracting a constant alone shifts exposure without reducing the ratio between positive endpoints. See Chapter 4.3, Encoding: linear, gamma or log.

A deblurring model must account for the domain in which the physical blur occurred (Figure 6.8.1⛶ fullscreen). An encoded-domain solver can be valid if it includes the nonlinear encoding in its forward model. Poisson blending instead chooses a gradient-domain appearance objective, which can legitimately differ across spaces (Figure 8.2.1⛶ fullscreen).

There is a further wrinkle: the values in a finished JPEG, or a scanned film frame, are usually not even a clean gamma encoding of scene light. The camera's processor (or the film's chemistry) has already applied tone curves, saturation boosts, and contrast S-curves to make the picture look good straight out of the camera: beautification, not measurement. So a consumer JPEG is roughly: scene light → camera "look" → gamma → 8-bit, with the physical meaning of the numbers scrambled along the way.

State the encoding and color space at every stage. Decoding sRGB recovers display-linear RGB; it does not undo a camera’s tone mapping to recover scene radiance. Raw development also needs black-level, white-level, and camera-color calibration.

Store encoding, color space, range, and alpha convention with the array. A conversion must update both the numbers and their description. Relabeling an sRGB array as linear does not decode it, and an integer-to-float cast does not perform a color conversion. Validate operation inputs and make conversions explicit when intent or information loss is involved.

Figure 8.2.2⛶ fullscreen distinguishes a deliberate look from an encoding mistake. A neutral or punchy rendition can be intentional; double encoding is a different error. Chapter 45.15, Rendering a raw DNG — from sensor numbers to a faithful image develops the raw-data interpretation in detail.

A black-level error can lift shadows and introduce a color cast before any tone adjustment (Figure 8.2.3⛶ fullscreen). Subtract the calibrated offset in the same units as the decoded raw samples. An image histogram with no values near zero is a diagnostic clue, not a measurement of black level: the scene may contain no dark surface, and flare can add real light. In this book's own case (the figure), the file declared a black level of zero while nothing in the data sat below about 0.18 of full scale, with the highest floor in the green channel, which is where the cast came from. Prefer trusted calibration, masked sensor pixels, or matched dark frames to subtracting the image minimum.

A useful programming analogy is to make the image representation part of its type. A buffer tagged as linear sRGB and a buffer tagged as encoded Display P3 should not be interchangeable merely because both contain three floats per pixel. A physical-light blend could require linear inputs; a color conversion could declare its source primaries and reference white. The conversion then becomes an explicit operation rather than an assumption hidden in a function name.

Such a type system would not choose the intended rendering or infer missing metadata, but it could catch a large class of mistakes: adding images with different representations, applying a transfer function twice, or treating a log value as a linear intensity. In ordinary array code, carry those assumptions as metadata and check them at boundaries. That is manual type checking for the image pipeline.

The black-level example came from a practical failure in this book's raw developer. A dusk photograph initially appeared milky with a green cast despite otherwise plausible processing. The supplied zero black-level value was accepted without checking whether it described the decoded samples. An offset mismatch early in the pipeline can survive every later stage and look like a poor tone or white-balance choice. The example motivates checking the decoder's units and offset handling, together with calibration; the minimum value in that photograph alone cannot prove the correct subtraction.

8.2.2 What the numbers actually mean: color spaces🔗⧉

Those three letters, "RGB," hide more than they reveal. Several independent decisions all travel under the same name, which is why two arrays can both be rightly called "RGB" and still be incompatible. The decisions that vary:

A bare array announces none of this: the information lives in metadata (the file header, an embedded ICC color profile). The advice is correspondingly simple: read the metadata, don't assume. When two images that "should" match stubbornly don't, this table is the first place to look.