BookNotesSlidesP-sets Syllabusdraft · v0.2.49.1
◀▲⌂▶
💬Comments welcome. To leave a note, select any text and click the note / highlight button that pops up — or open the panel with the tab at the top-right (‹). Notes are visible only inside our private review group.
💡 In a hurry? Jump to this chapter’s 4 big lessons ↓

4.3 Encoding: linear, gamma or log⧉

Linear image values are proportional to the measured or represented light. An encoding maps them to stored numbers. The choice matters both for precision and for processing: adding encoded values generally does not add the corresponding light.

Uniform integer steps in linear light devote relatively few levels to dark regions. Human contrast sensitivity often depends more closely on relative than absolute differences, so a nonlinear encoding can use a limited bit budget more effectively Poynton 2012. This is an engineering approximation, not a universal law of brightness. Visibility also depends on the background, spatial structure, noise, and viewing conditions.

For example, a range of 100:1 divided into successive one-percent ratios needs about $\ln(100)/\ln(1.01)\approx463$ intervals. This motivates allocating codes by relative precision, but it does not prove that eight bits always suffice. Smooth gradients and subsequent editing can expose eight-bit quantization.

The name “gamma correction” encourages the wrong mental model. An encoding changes the numbers used to represent light; decoding reverses that change. There may also be a display nonlinearity to compensate and a deliberate contrast change to apply, but these are separate jobs even when all three involve power curves. Poynton's discussion emphasizes the continuing value of perceptual encoding after the original CRT motivation Poynton 2012. Removing an old display technology does not remove the need to allocate a finite code range according to visibility.

4.3.1 Gamma encoding⧉

For a simple power encoding on $[0,1]$,

$$V=L^{1/\gamma},\qquad L=V^\gamma.$$

A value such as $\gamma=2.2$ moves code precision toward the shadows. It does not make all steps perceptually equal, and it is not the exact sRGB curve. I prefer “encode” and “decode” to the ambiguous “gamma correction”: name the direction and the actual transfer function.

fig-quantization-banding
Figure 4.3.1. Gamma and quantization, together. The 2<sup>N</sup>-bit code values are placed by the gamma curve (left panel): with γ&gt;1 they bunch in the shadows, where the eye is most sensitive to ratios. A smooth gray ramp, an all-hue color gradient, and a real photograph are each quantized through that curve. Set γ = 1 (linear) at a low bit depth and the shadows band badly while the highlights stay smooth (precision in the wrong place); raise γ toward 2.2 and the same bit budget spreads the steps out perceptually, hiding them. The pipeline is store (encode → quantize → decode) then re-encode to sRGB for the screen; the ramp is perceptually uniform and the gradient even in CIELCh, so the only visible structure left is the quantization itself. Interactive: slide gamma γ and the bit depth N (default 4 bits, γ = 2.2); pick or upload the photo.
fig-jnd-per-step
Figure 4.3.2. How many just-noticeable differences is one code step worth? (interactive) A code step is a jump in linear light, and Weber's law says the smallest visible jump is a roughly constant fraction of the level, so a step is worth $\Delta L/(k_w(L+L_0))$ JNDs and a bar above 1 is a step you can see as a band. At $\gamma = 1$ the bars are a cliff: shadow steps are worth hundreds of JNDs while highlight steps fall under threshold, so codes are wasted where the eye cannot use them and starved where it can. Raising $\gamma$ does not remove the problem, it levels it once the display has a black level: with $L_0 = 10^{-3}$ of white, at 8 bits the worst step falls from about 390 JNDs to about 11, while more steps end up marginally over threshold. Under strict Weber's law ($L_0 = 0$, the figure's default) the first step, up from zero light, is infinite at any $\gamma$, and the next worst step is larger at $\gamma = 2.2$ (about 360 JNDs) than at $\gamma = 1$ (100), so the leveling depends on the black level. Evening out the error is the whole reason images are stored gamma-encoded. The model is illustrative, not a prediction for a particular display. The 1% threshold is a best case, measured on large fields with sharp edges; a smooth gradient is judged less harshly, which is why 8-bit images band mildly rather than everywhere at once.
fig-display-gamma-probe
Figure 4.3.3. What code value do you have to send to get half the light of white? The left half of the strip is a one-pixel black-and-white checker: half its area is off and half is full on, and light adds, so it emits exactly half the light of white whatever the system does to code values. The right half is solid, at a code value you set. Back away until the checker fuses into flat gray, a meter or so, or squint, then slide until the boundary between the halves disappears. On anything behaving like sRGB the answer lands near code 186 of 255, nowhere near the halfway code of 128, and $0.5 = (V')^{\gamma}$ turns it into an exponent near $2.2$. The bars above say the encoding has to bend; this says by how much, on the screen in front of you. What is measured is not the panel alone but everything between the number and the light: the browser, the OS compositor, whatever calibration curve the GPU has loaded, and then the panel. Hover the strip to magnify the device pixels, or raise k to make the checker plainly a checker. Reset the browser zoom to 100% first: a scaled canvas resamples the checker and averages it in the wrong domain.

For sRGB, per channel,

$$ \text{OETF} : \mathrm{RGB} \to R'G'B', \qquad R' = \begin{cases} 12.92\,R, & R \le 0.0031308 \\[2pt] 1.055\,R^{1/2.4} - 0.055, & R > 0.0031308, \end{cases} $$
$$ \text{EOTF} : R'G'B' \to \mathrm{RGB}, \qquad R = \begin{cases} R'/12.92, & R' \le 0.04045 \\[2pt] \big((R' + 0.055)/1.055\big)^{2.4}, & R' > 0.04045, \end{cases} $$

and identically for $G$ and $B$. The prime does real work: $R'$ is what ships in the file, and $R$ is what you must convert to before any linear-light arithmetic. In code the pair is short, and worth keeping named so a caller can never confuse the two directions:

fig-srgb-gamma-curve
Figure 4.3.4. The piecewise sRGB encoding, up close. The left panel plots the full standard curve on $[0,1]$ (the linear toe $V = 12.92\,L$ splicing into the power segment $V = 1.055\,L^{1/2.4} - 0.055$) against a light dashed pure power $V = L^{1/2.4}$ for reference. The right panel zooms into the origin (drag the slider) until the straight linear segment, the breakpoint at $L = 0.0031308$, and the small slope kink where the two pieces meet all become visible.
Listing 4.3.1 · sRGB encode / decodeembed
# sRGB OETF / EOTF, per channel, on values in [0, 1].
def srgb_encode(lin):        # RGB (linear) -> R'G'B' (gamma)
    return 12.92*lin if lin <= 0.0031308 else 1.055*lin**(1/2.4) - 0.055

def srgb_decode(enc):        # R'G'B' (gamma) -> RGB (linear)
    return enc/12.92 if enc <= 0.04045 else ((enc + 0.055)/1.055)**2.4

These snippets assume finite values in $[0,1]$. Working RGB may contain negatives or values above one, so a production implementation needs a documented extended-range convention; silently clipping before processing loses information. The rounded sRGB encoding junction differs by about $3\times10^{-8}$ in code value, not $10^{-4}$. Use the published constants consistently.

For a pure power curve at $N$ bits, the smallest positive decoded value is $(2^N-1)^{-\gamma}$. Its distance below white is $\gamma\log_2(2^N-1)$ stops. This describes the smallest positive code, not usable sensor dynamic range, and it does not describe sRGB’s linear toe. Eight-bit sRGB’s first positive code is about $1/(255\times12.92)$, roughly 11.7 stops below one. Neither count says how well intermediate tones are resolved or how far above noise a signal lies.

fig-encoding-dynamic-range
Figure 4.3.5. Dynamic range from a bit budget, and why noise undercuts it. Choose the bit depth N and the gamma γ. The readout gives the smallest nonzero level and the dynamic range in stops for linear encoding (about N stops) and for gamma encoding (about γ·N stops). The curve draws each encoding as code value versus light on a log (stops) axis, and the two tick rows show where the 2<sup>N</sup> code levels fall: linear levels crowd against white and bottom out near −N stops, while gamma levels reach far deeper into the shadows. Those deep-shadow stops are an upper bound only, since noise sets the usable floor.

Historically, a CRT’s approximate power-law light output performed the decoding. Modern display systems reproduce a specified response through their electronics and calibration. Do not collapse all standards to “gamma 2.2”: Adobe RGB (1998) specifies a power near 2.2, ProPhoto/ROMM RGB uses 1.8 with a toe, and Rec. 709’s camera encoding is distinct from the BT.1886 reference display response, whose nominal exponent is 2.4. Apple's Macintosh used a system gamma of 1.8 for years, until Mac OS X 10.6 moved to 2.2 in 2009, which is one reason older untagged files can look too light or too dark on the other platform.

Film gamma is another quantity: the slope $\Delta D/\Delta\log_{10}H$ of optical density against log exposure in the straight part of a characteristic curve. It describes film contrast. An end-to-end scene-to-display rendering can intentionally change contrast; its transfer need not be the identity even when every encoding stage is correctly inverted.

💡 Big lesson (L4.4) — match the display interface’s expected encoding

A typical sRGB image file contains encoded values. Decode them for linear-light calculations, then supply the representation expected by the output interface. A viewer expecting sRGB needs sRGB-encoded output; a linear rendering surface may encode automatically. Encoding twice is as wrong as omitting it. RAW, log, HDR, and floating-point files require their own declared interpretation.

The application, operating system, GPU, and display share the output conversion. An embedded profile helps define the input, but it does not guarantee every application will honor it. Test the complete path rather than inferring it from the file extension.

For Rec. 709/sRGB primaries, relative luminance is $Y=0.2126R+0.7152G+0.0722B$ on linear values. Rec. 709 luma uses those coefficients on its nonlinear components. A common Rec. 601-style luma used with JPEG is instead

$$Y'=0.299R'+0.587G'+0.114B'.$$

The coefficients and the encoding are independent pieces of the definition. Rec. 709 coefficients are not reserved for linear light.

fig-luma-vs-luminance
Figure 4.3.6. Luma is not luminance, and this is the shape of the difference (interactive). The sRGB gamut drawn in CIE $XYZ$, cut into sheets of equal luminance $Y$, the Rec. 709 weights on linear light, and of equal luma $Y'$, the same weights on the gamma-encoded numbers, which is what JPEG and Y'CbCr carry. Show either stack or both: their levels are matched on the neutral axis, a luma sheet at $Y' = V$ against the luminance sheet at $Y = \mathrm{eotf}(V)$, so the two meet exactly on the grays and everything else you see is the buckling. Luminance sheets lie flat, since $Y$ is an axis of the space; luma sheets buckle, because a plane in the encoded cube is a curved surface once the decode has bent it. The cost is printed beside them: across the $Y' = 0.5$ sheet, every color of which a codec calls equally bright, the light runs $0.214$ to $0.338$, a factor of $1.6$; across the luminance sheet through the same grays, luma runs $0.226$ to $0.500$, a factor of $2.2$. Drag to orbit; the $X$, $Y$, $Z$ axes are drawn from black.
💡 Big lesson (L4.5) — luma is not luminance

Luma is a weighted sum of specified nonlinear RGB components. Luminance is a photometric quantity obtained from linear values with the appropriate primaries and scale. Weighting and nonlinear encoding generally do not commute. Neither constant luma nor constant luminance guarantees equal perceived brightness across colors.

A simple level-counting calculation explains the attraction of nonlinear storage. Suppose, for this example, that adjacent displayed luminances should differ by at most one percent and that the brightest level is 100 times the darkest. Repeated multiplication by 1.01 needs about 463 intervals to span that range. At two percent, about 233 intervals suffice. These are illustrative constant-ratio thresholds, not a full visibility model: adaptation, spatial structure, and the display black level matter.

Compare that allocation with linear codes. Moving from code 1 to code 2 doubles the light; moving from 254 to 255 changes it by less than half a percent. The same absolute increment therefore buys very different relative precision. A compressive encoding allocates more codes to small linear-light values. Eight bits can still show banding, especially on smooth ramps, and a power law is not exactly the geometric spacing of a logarithm.

Transmission noise gives the encoding another job. Suppose approximately equal-sized errors are added to the encoded signal on its way to a receiver. Encoding has expanded the small linear-light values before that noise is added. Decoding then compresses an encoded error near black into a smaller absolute light error than the same encoded error near white. This is noise shaping, analogous to allocating quantization precision unevenly. It does not eliminate noise or guarantee constant visibility. The order is essential: noise added by the sensor before encoding has already affected the measurement, whereas channel noise added after encoding is subjected to the decoder's local slope.

The CRT joined this communication problem to a device constraint. Its approximate power response could perform the receiver's decoding as part of producing light. A modern system may distribute that job across application conversion, graphics hardware, display electronics, and calibration. The required input convention remains what matters to the programmer. Sending linear values to a path that interprets them as sRGB-encoded values applies the wrong relation between number and light, even if no individual component is broken.

A long pipeline also creates an ownership problem. The editor may convert a buffer that the operating system then converts again, or each may assume the other will perform a required conversion. The result is a double conversion or a missing one. An embedded profile is useful only if it still describes the numbers in the buffer and the next stage reads it correctly. A machine-checkable buffer type describing encoding, primaries, and image state would make these interface errors easier to catch. Keeping those labels explicit is the manual equivalent.

4.3.2 Gamma the contrast operator (gamma adjustment)⧉

There is a second, entirely different thing called "gamma," and conflating it with the encoding is one of the classic color confusions. The gamma adjustment (the "gamma" slider in an editor) is a tone operator: it applies a power curve $V_\text{out} = V_\text{in}^{1/g}$ to the image to brighten or darken the midtones while pinning black and white, a midtone-contrast control. It has the same mathematical form as encoding gamma, a power law, which is exactly why the two get confused, but it is not the same thing: encoding gamma is an invertible change of representation before quantization; gamma adjustment deliberately changes the picture while leaving its encoding convention in place. A power edit can itself be inverted before clipping or quantization.

You meet it most famously as the middle slider in Photoshop's Levels dialog (Figure 4.3.7), the one between the black-point and white-point sliders. Levels maps input to output through a black point $b$, a white point $w$, an output range $[V_\text{lo}, V_\text{hi}]$, and a midtone gamma $g$:

$$ V_\text{out} = V_\text{lo} + (V_\text{hi} - V_\text{lo})\left(\frac{\operatorname{clamp}(V_\text{in},\, b,\, w) - b}{w - b}\right)^{1/g}, $$

with $w>b$ and $g>0$. Here $b, w$ set the input black and white points, $V_\text{lo}, V_\text{hi}$ the output range (usually $0$ and $1$), and $g$ (the middle slider, default $1$) is the midtone gamma: $g>1$ lifts the midtones, $g<1$ darkens them, both leaving the endpoints fixed. You can watch this exact control operate in the two ports of Photoshop 1.0 we ship, the original 1990 binary in an emulator and our line-by-line JavaScript re-implementation (Chapter 45.33, Photoshop 1.0, running, Chapter 45.34, Photoshop 1.0 ported to JavaScript), where the Levels code is a handful of lines and the gamma slider is literally an exponent. The lesson: when a tool says "gamma," ask which gamma, the invisible encoding you must respect, or the visible contrast knob you are turning on purpose.

fig-photoshop-levels
Figure 4.3.7. Levels, live, and the other gamma (interactive). Photoshop 1.0's Levels, running the map the 1990 source computes and our line-by-line port still computes (Chapter 45.34, Photoshop 1.0 ported to JavaScript): clamp to the black and white points, raise to the power $1/g$, then map into the output range, all on code values rather than on light, which is what makes it a contrast control and not an exposure one. The middle triangle is the gamma adjustment. It moves the midtones and leaves both endpoints fixed, a deliberate change to how the photograph looks; encoding gamma has the same formula and the opposite status, a container you decode and re-encode without touching the picture at all. The histogram is the picture's own and follows it as you switch photographs. The second one is where the cost shows: pull the black and white points in and the surviving codes are stretched apart, leaving gaps the picture cannot fill (at the settings it opens with, 200 of 256 code values still carry pixels), which is banding waiting to happen and the reason a heavy Levels move wants more than eight bits under it.

4.3.3 System gamma: the end-to-end curve⧉

A third use of the word describes the whole chain rather than one stage. The end-to-end, or system, gamma is the slope, in log-log coordinates, of the curve from scene luminance to displayed luminance in the middle tones. If the display exactly undid the camera's encoding, it would be one, and the display would reproduce the scene's luminance ratios. Systems that look right use a system gamma above one, about 1.2 for a television in a dim room and up to 1.5 for a cinema in the dark, because a picture is seen under different conditions from the scene: a darker surround, a far lower absolute luminance, and flare in the room all make a faithful copy look flat. The system gamma is there to account for those viewing conditions; Chapter 18.1, Tone mapping and perception explains each effect.

The CRT made this convenient. Television cameras encoded light with an exponent of about 0.5 (the Rec. 709 camera curve is close to a square root), and a CRT's electron gun produces light roughly as the 2.4 power of its input. The display therefore did more than undo the encoding: the product of the two exponents, about 1.2, is the system gamma a dim living room needs. The physics of the tube happened to supply both the decoding and the perceptual tone mapping, which is one reason the BT.1886 reference display kept an exponent of 2.4 after CRTs disappeared.

4.3.4 Log encoding⧉

For positive light, logarithms turn exposure ratios into differences:

$$V=\log_2(L/L_{\rm ref}),\qquad \log_2(aL/L_{\rm ref})=V+\log_2a.$$

One stop is an increment of one. This is useful for exposure edits, relative contrast, and models of multiplicative illumination. Log values shift under a common gain; they are not themselves invariant. Differences between log values are invariant to that common gain.

Zero and negative values need an explicit policy. For a diagnostic log image, choose a positive floor $\epsilon$ in the same units as $L$, record which values were floored, and keep the reference scale fixed across images:

require epsilon > 0 and L_ref > 0
require all input values are finite
floored = (L < epsilon)
V = log2(max(L, epsilon) / L_ref)
# On unfloored samples only: L = L_ref * 2**V

A floor based on the smallest positive pixel varies with noise, crops, and frames, and fails on an all-black image unless separately handled. Flooring also destroys signed values. Camera log curves use specified toes and offsets; they are not this generic logarithm. Decode the named curve and range, such as LogC, S-Log3, or V-Log, before a scene-linear calculation. ACEScc and ACEScct are grading representations within ACES, alongside its linear representations.

fig-encoding-curves-compare
Figure 4.3.8. Four storage curves and one quantizer (interactive). Linear, gamma, log and PQ storage drawn together as curves from linear light to stored value, with the N-bit staircase of one of them. Gamma and log bend the curve up to give the shadows more codes, log most of all; PQ encodes absolute luminance, so it needs a white, and with a white below 10,000 nits its top codes go unused. Interactive: choose the stepped curve, the bit depth, the storage gamma and the log range, include PQ and set its white, and switch to a log light axis.
fig-quantization-banding-log
Figure 4.3.9. Linear, gamma and log storage with quantization (interactive). The pipeline of Figure 4.3.1, with log storage added: a gray ramp, a color gradient and a photograph are taken to linear light, stored at N bits with the chosen encoding, and shown again in sRGB. For log storage, each image's zero pixels are set to half its smallest non-zero pixel, and the codes are spread evenly in ratio from that floor to white. The floor is reported for each image, and it differs between them, so the same bit depth covers a different number of stops on each. At low bit depth log banding is even from shadow to highlight, but some of its codes go to shadows darker than the display can show. Interactive: choose linear, gamma, log or floating-point storage (fp8 in its E4M3 and E5M2 forms, and fp16, each value rounded to the nearest number the format holds), set the bit depth and the storage gamma, take the log range from each image, set it in stops or match γ's step from code 1 to 2, show the JNDs per code step, and change the photograph.
fig-jnd-per-step-log
Figure 4.3.10. How many just-noticeable differences is one log code step worth? (interactive) The chart of Figure 4.3.2 with log storage added. With no image to take a floor from, the rule is applied to an 8-bit sRGB source: zero becomes half of its smallest non-zero pixel, 1.5 × 10⁻⁴, about 12.7 stops below white. Every log step is then the same ratio of light, so under strict Weber's law, the default here, every step is worth the same number of JNDs: 3.5 at 8 bits and 0.86 at 10 bits. Under the same law a power law is uneven. At γ = 2.2 the step from code 1 to code 2 is worth about 360 JNDs and 220 steps exceed one JND at any bit depth, while the upper steps fall well below it. A black level L₀ adds a floor to the threshold, as in Figure 4.3.2, and the log steps below it are then worth less than the rest. Interactive: switch between power-law, log, PQ and floating-point storage (fp8 in its E4M3 and E5M2 forms, and fp16, whose codes are every value the format holds between 0 and 1), set the bit depth, the storage gamma, the log range, the PQ white and the black level, and compare against γ = 1 or γ = 2.2.

Homomorphic filtering and Retinex Land & McCann 1971 use logarithms to separate multiplicative factors under image-model assumptions. A high-pass log filter can suppress slowly varying illumination, but material color can vary slowly and shadows can have sharp boundaries. This is not a general recovery of reflectance.

Other uses include log-chromaticity methods, the Chong et al. space Chong et al. 2008, DIN99’s perceptual compression, and LogLuv image storage. Their objectives and definitions differ. Photographic density is $D=-\log_{10}T$ for transmittance $T$. Cineon represents printing density; its code increments are not scene-exposure stops without accounting for the film characteristic slope.

💡 Big lesson (L4.6) — additive vs multiplicative → choice of encoding

Use linear light for an operation that adds light. Use ratios or log differences when the model concerns multiplicative changes. Choose a perceptual encoding for storage precision or an edit defined in that representation. Log sharpening can be a legitimate contrast edit; it simply implements a different operator from sharpening linear light.

The following implementations expose the floor and reference value as explicit parameters. They implement the diagnostic mapping above, with a common contract: samples and parameters are finite, epsilon and reference are positive, and values below epsilon are replaced before the logarithm. The reference fixes the zero of the stop scale. Computing the difference of logarithms avoids overflowing the intermediate ratio for a very small reference.

Listing 4.3.2 · linear light -> log2, with an explicit floorembed
import math

def to_log2(values, epsilon, reference=1.0):
    if not (math.isfinite(epsilon) and epsilon > 0 and
            math.isfinite(reference) and reference > 0):
        raise ValueError("epsilon and reference must be finite and positive")
    offset = math.log2(reference)
    result = []
    for value in values:
        if not math.isfinite(value):
            raise ValueError("samples must be finite")
        result.append(math.log2(max(value, epsilon)) - offset)
    return result

For example, with reference=1 and epsilon=1/256, samples 1, 1/2, and 1/4 map to 0, -1, and -2 stops. A zero maps to -8. It can no longer be distinguished from a true 1/256 sample unless we retain the floor mask; the logarithm does not make discarded information recoverable.

The illumination–reflectance model also explains why logarithms recur in image processing. If an image is approximated by the product of an illumination field and a reflectance field, its logarithm is the sum of their logarithms. A spatial filter can then operate on the two contributions additively. Homomorphic filtering estimates a slowly varying component in log space, reduces that component, and exponentiates. Fine reflectance changes can retain their local contrast while broad brightness variation is compressed. Unfortunately, a broad material region can enter the low-frequency component, and a sharp shadow can enter the detail component. The decomposition is useful without being an exact physical separation.

Retinex Land & McCann 1971 connects comparisons across an image to color constancy. A ratio of two positive intensities becomes a difference of their logarithms. A common multiplicative gain cancels in that difference, which explains the appeal of relative comparisons. Spatially varying illumination does not cancel so simply. This connects log processing to Chapter 3.3, Adaptation and constancy and to the base/detail methods in Chapter 18.3, Two-scale and bilateral tone mapping.

Log recording and display encoding serve different stages. A named camera log curve can hold a broad range of scene values while making an exposure ratio correspond approximately to a code increment. That is convenient for grading. If those code values are sent directly to a display expecting sRGB, the picture usually looks flat because the intended rendering and display conversion have not happened. The flat appearance is not the goal of log capture. Decode the specified representation or use its defined rendering transform, then prepare the result for the target display.

Film density provides an older version of the same convenience. Multiplying transmittance changes optical density by an additive amount. Cineon's printing-density codes made that quantity available to digital grading. Converting a density increment into a scene-exposure increment also needs the film characteristic slope. For an illustrative straight-line slope of 0.6, one exposure stop changes density by about 0.6 times 0.301, or 0.181. If codes are spaced by 0.002 density units, that is about 90 code intervals per stop. Outside the straight part of the characteristic curve, the same calculation no longer applies. This accounts for the roughly constant-stop behavior without treating density itself as log scene exposure.

4.3.5 PQ, the perceptual quantizer⧉

PQ (SMPTE ST 2084, used by Rec. 2100) maps a normalized code $V$ to a nominal absolute display luminance $L_d$ up to 10,000 cd/m²:

$$L_d=10000\left[\frac{\max(V^{1/m_2}-c_1,0)}{c_2-c_3V^{1/m_2}}\right]^{1/m_1},$$

where $m_1=2610/16384$, $m_2=2523/32$, $c_1=3424/4096$, $c_2=2413/128$, and $c_3=2392/128$, for $0\le V\le1$. For RGB, this transfer applies to the component signals; photometric luminance also depends on their combination.

PQ allocates precision using a contrast-sensitivity model. Ten-bit output does not guarantee invisible banding for every gradient, processing history, or viewing condition. A display may tone-map content outside its capabilities, so the specified luminance is a reference target rather than a promise about the actual emitted light. PQ can be paired with RGB and several component representations; it is not restricted to ICtCp.

4.3.6 HDR encoding⧉

HDR requires a sufficient numeric range and a defined interpretation of that range. The data type and transfer function do not, by themselves, determine whether an image is scene-referred or display-referred.

Floating-point scene-linear images are convenient for rendering, merging, and compositing. IEEE half precision has finite range: its positive normal values extend from $2^{-14}$ to 65504, about 30 stops. Subnormal values extend down to $2^{-24}$ with decreasing relative precision. OpenEXR supports half and other channel types. Values above one are ordinary working values; infinity is not valid unlimited image headroom. Radiance RGBE uses a shared exponent for three channels, reducing precision for small components beside a large one.

For HDR delivery, Rec. 2100 specifies PQ and HLG systems. HLG’s scene-light encoding combines square-root and logarithmic segments; its display system includes a viewing-dependent scene-to-display transform. It is relative, but it does not simply stretch every signal linearly to the panel peak. SDR compatibility is conditional on the playback path and does not mean every unconverted SDR display reproduces the intended picture.

Gain-map formats combine a base rendition with encoded gains and metadata for reconstruction at a different display headroom. The reconstruction applies the specified gains in the prescribed linear domain, with any offsets and weighting defined by the format. A gain map can vary in resolution and may have one or several channels. ISO 21496-1 standardizes this kind of representation. An older reader may show the base rendition while a supporting reader reconstructs HDR. Test both paths and preserve the metadata.

A scene-linear image still needs rendering for a particular display. Transfer encoding alone does not perform tone mapping or gamut mapping; see Chapter 18.2, Global tone mapping.

Floating point combines two useful properties. Its represented value is linear, so addition can still model light addition, while its exponent makes the spacing grow with magnitude. Within a normal exponent bin, the significand supplies the fine steps; moving to the next bin doubles both the scale and the step size. That is why a floating-point working image can cover a large range at approximately constant relative precision without applying a logarithm before every operation. A shared-exponent RGB format saves storage by choosing one scale for all channels, but a small component then has fewer useful steps beside a much larger one.

4.3.7 When to work on encoded values⧉

Choose the representation from the operator’s definition. Optical blur, radiometric exposure scaling, and physically based compositing require linear light. Quantization and compression often benefit from a perceptually motivated encoding. Tone curves can be deliberately defined on encoded values; that choice is part of the edit.

For denoising, use the noise model for the actual input domain. sRGB encoding is not a general variance-stabilizing transform. A square-root-like transform can approximately stabilize shot noise, but read noise, clipping, demosaicking, and later processing change that argument. The noise chapter derives the applicable models; do not import a uniform-noise denoiser merely because an image has been gamma-encoded.

4.3.8 Color and encoding in real pipelines⧉

A pipeline's choice of color coordinates determines what its operations mean. The same library can reconstruct radiance in one application and blend encoded samples in another. Follow the values through the implementation: where they become proportional to light, where an appearance transform changes them, and what the next stage assumes.

Consider an ordinary sRGB JPEG loaded into an array. Dividing its integer samples by 255 changes the numerical range; casting them to floating point changes the precision. Neither operation undoes the sRGB transfer function. A convolution will run on either array, and often produce a plausible picture. The programmer must supply the interpretation that makes its arithmetic meaningful.

The difference can occur within one library. OpenCV's detailed stitching sample loads images with imread, warps them, applies gain compensation, and feeds the result to a multiband blender. It does not decode a transfer curve along that path. With sRGB JPEG input, those operations therefore use encoded samples. Its HDR example, on the other hand, calibrates a response with createCalibrateDebevec, passes that response to createMergeDebevec, and tone maps the recovered HDR image. Response calibration and merging are separate calls. A convincing panorama and a radiance estimate impose different requirements on the same library.

A camera pipeline also has several reasons to change coordinates. Sensor corrections use calibrated measurements; later sharpening or chroma denoising may deliberately use encoded luma and chroma. Raw editors can provide perceptual controls between scene-linear stages. These choices are part of the rendering design. For a concrete example, darktable's documented scene-referred workflow keeps as much processing as possible in linear RGB and places tone compression late. Its display-referred workflow moves that boundary earlier. Even within the scene-referred workflow, a module can transform to another representation internally. “Scene-referred” describes the interpretation of an image, not a promise that every operation is linear.

RawTherapee's toolchain description makes the changes of coordinates visible: camera calibration and working RGB lead into operations that use Lab, followed by a conversion back to RGB for output. Its RGB and Lab comparison shows that similarly named saturation controls can produce different results because they operate on different coordinates. The working RGB profile alone does not specify the behavior of every photographic control.

The openISP reference implementation includes both early sensor processing and later luma/chroma filtering, edge enhancement, and hue/saturation controls. Such a design allocates different jobs to different domains. A luma denoiser after encoding does not have the same noise distribution as a denoiser on sensor measurements. These examples explain why a universal prescription to linearize an entire pipeline, or to keep the whole pipeline encoded, misses the purpose of individual stages. The detailed ISP discussion continues in Chapter 8.12, Recap ISP, non-destructive editing.

Panorama software provides another instructive comparison. Hugin's photometric optimizer models exposure, vignetting, white balance, and camera response. Its EMoR response model represents a curve with five coefficients. Matching overlapping scene points then involves correcting how the camera turned light into code values, rather than applying one gain to those code values and assuming that it represents exposure. The fitted model still depends on the images and the parameters chosen for optimization.

Enfuse has a different objective. It combines exposures using local measures of exposure, saturation, and contrast, based on the exposure-fusion method of Mertens, Kautz, and Van Reeth. It can produce a useful display image without first recovering a radiance map. Its success should therefore be judged as an appearance result. HDR reconstruction and exposure fusion can start from the same bracket but solve different problems; that difference explains why their treatment of camera response need not be the same.

Deblurring exposes the physical assumption especially clearly. A lens mixes incoming light before the camera encodes it. Encoding that mixture and blurring an already encoded image give different results, even without clipping. A reconstruction based on an optical convolution must account for this order. A learned model trained on stored RGB pairs can instead learn a mapping in that storage domain, but the training preprocessing then becomes part of its specification. Changing the test encoding changes the problem the model sees.

Graphics makes the conversion boundary explicit because light transport adds radiance. For example, the image implementation in Physically Based Rendering associates images with a color encoding and provides separate conversions to linear values and back. An encoded reflectance texture needs decoding before lighting. A normal map has vector components rather than encoded colors, even when both assets happen to arrive in the same file format.

A two-color mixture makes the error visible without a whole renderer. Equal amounts of linear sRGB red and green give $(0.5,0.5,0)$ in linear RGB, about $(0.735,0.735,0)$ after sRGB encoding. Averaging the encoded endpoints instead gives $(0.5,0.5,0)$ in code values, only about $(0.214,0.214,0)$ in linear light. In this symmetric example the chromaticity is unchanged but luminance is much lower. For general endpoint colors, the channel ratios can change too. A perceptual palette gradient can deliberately follow another path; a simulation of two mixed lights has this specific linear answer. The cross-fade figures in Chapter 4.9, Color TLDR compare these paths on patches and photographs.

4.3.9 What gamma buys, and what it costs⧉

Nonlinear encoding redistributes precision. It can also reshape the effect of noise added after encoding, as in a transmission channel. It does not recover sensor information, guarantee uniform noise, or make a power law invariant to illumination. If $V=L^{1/\gamma}$, multiplying $L$ by $a$ multiplies $V$ by $a^{1/\gamma}$; only a logarithm converts that multiplication into an additive shift.

The processing cost is that arithmetic changes meaning. Averaging encoded black and white and then sRGB-decoding gives about 0.214 units of light; averaging their linear values gives 0.5. A log average gives a geometric mean, with a result near black that depends on the floor. These are different operators, not three interchangeable implementations.

fig-encoding-1d
Figure 4.3.11. One filter, three encodings (interactive). A one-dimensional signal is encoded three ways, filtered in each, and decoded back to light, which is the only place the three answers can be compared. The rows are the input in that encoding's own units, the output there, the output back in light with the linear answer behind it in gray, and the difference from the linear answer with its RMS. The first column's difference is identically zero, which is the check that the machinery is doing what it claims. Signals: a step, steps of growing height, a sine, white noise, or a scanline from a photograph, with noise added in linear light, where a sensor makes it. Methods keep their own knobs: box radius, Gaussian $\sigma$, unsharp radius and amount, downsample factor, bilateral $\sigma_s$ and $\sigma_r$. The bilateral filter is where the encoding bites hardest, and it bites in both directions. With noise $0.04$ and $\sigma_r = 0.06$, denoising a flat patch at 5% of white leaves 40% of the noise in linear light but 81% in gamma and 83% in log, because stretching the shadows makes the same numeric $\sigma_r$ far shorter than the noise it is supposed to bridge. Move the patch to 50% of white and it reverses: 44% in linear against 33% and 27%. Same filter, same numbers, opposite verdict.
fig-encoding-lab
Figure 4.3.12. Where the arithmetic happens, for ten operations (interactive). Four panes over one picture: three of them run the same operation in three spaces, light, gamma and log, and the fourth is the input. Every pane is displayed identically, decoded back to light and then sRGB-encoded for the screen, so anything visible between them is the encoding rather than the display. Move the mouse over the picture to slide the splits, scroll to zoom, drag to pan, double-click to reset; A and B sit above the picture, C and D below, and each pane's buttons choose what it shows. What the panes are scored against depends on the operation, and the figure says which. Some of these operations are trying to recover something we have, and for those there is a real ground truth: denoising, deblurring (the blur is applied inside the lab), super-resolution, and the resample round trip, which is supposed to give the picture back. For the blur and the downsampling, the degradation is generated separately in each working space, so each pane shows a whole round trip in that space rather than the inversion of one shared physical observation. Others have no truth but do have a definition, and the definition is in light: a blur, a sharpen, a gradient, a cross-fade, and for those the reference is the same operation done in light on the clean image, so with the noise at zero the linear pane matches it exactly (infinite PSNR) and the other two show the encoding error on its own. A corner detector has neither, so it is offered no reference and reported no metric. When the ground truth is the clean image, the PSNR measures what the operation cost rather than what the encoding did, and the two are not the same question: to see the encoding alone, set a pane to the difference between two panes and take gamma minus light. Turn the noise up and photon and read noise are added in light, the way a sensor makes them, and the panes begin to differ in how each space handles a noisy signal as well. The operations are blur, unsharp sharpening, Sobel gradient, iterative deblurring, resampling down and back up, reconstruction super-resolution, a cross-fade against a brighter frame, bilateral and pyramid denoising, and the Harris corner response; each brings its own canonical sliders. The inputs are four scene-linear photographs from the book's own DNGs, chosen for what the operations do to them (thin structure, fine texture, deep shadows, smooth gradients), and the four test patterns the Fourier and resampling chapters use, where the right answer is known by construction. Two things to try first. Take the zone plate and resample it down and back up: in light the fine rings average toward the mid gray they are made of, and in gamma the same average comes out visibly darker, which is Jensen's inequality with a picture attached. The direction is not a matter of taste: the decode is convex, so for an average with nonnegative weights, decoding the average of encoded values lands at or below the average of the light, with equality where the signal is locally flat or stays within the curve's linear toe. On a black-and-white pair the same average gives 0.50 in light, 0.21 in gamma and 0.05 in log. The kernel sets the size of it, because it sets how much averaging happens: on the zone plate the round trip preserves the mean light in linear and loses 9.0% of it in gamma with bilinear, 3.0% with box and 2.5% with bicubic, since a kernel with negative lobes averages less and so loses less. Lanczos-3 almost breaks even before clipping, and then loses it again to clipping, because ringing sends pixels past white: with bicubic, 1.3% of the gamma result and 8.8% of the log result overshoot and are cut off at the top. The deficit also grows with local contrast, which is why fine texture darkens and flat areas do not. The filter is yours to choose, box, bilinear, Catmull-Rom or Lanczos-3, and both directions use it, so the round trip is one filter's fault rather than two; the sharper kernels ring, and the ringing lands in different places in different spaces. Note also what the ground-truth PSNR says here: on a photograph it is about 27.5 dB in all three spaces, because what the round trip destroys is resolution, which no space recovers. That number is not measuring the encoding, and the difference view is. Then take the dark interior, turn the noise up, and denoise: here the parameters are the trap, since a range sigma of 0.08 means one thing in light and something else entirely in gamma, and the panes are what a naive port between spaces actually produces. A third, for detectors. Run Harris corners on the bridge. The response $R = \det - k\,\mathrm{tr}^2$ is a fourth power of image gradients, so its absolute value means nothing across spaces; what transfers is the ranking, and each pane keeps its own strongest 200 after non-maximum suppression, exactly as a real detector does. In light they land on the towers and cables against the sky; in log they swarm the dark hillside, because a log space is a space of ratios and the shadows are where the ratios are large. Same detector, same count, different picture: the space is part of the algorithm, not a display convention applied afterward.
💡 Big lesson (L4.7) — averaging colors belongs in linear light

To average light, decode, combine linear values, and encode for the output interface. For nonnegative weights summing to one and a convex decoding curve, averaging encoded values produces no more light than averaging the decoded inputs. Kernels with negative weights need separate analysis. For alpha compositing, linearize color and use the correct premultiplied-alpha convention; alpha itself is not gamma-encoded color.

Recap: big lessons of this chapter

💡 Big lesson (L4.4) — match the display interface’s expected encoding

A typical sRGB image file contains encoded values. Decode them for linear-light calculations, then supply the representation expected by the output interface. A viewer expecting sRGB needs sRGB-encoded output; a linear rendering surface may encode automatically. Encoding twice is as wrong as omitting it. RAW, log, HDR, and floating-point files require their own declared interpretation.

💡 Big lesson (L4.5) — luma is not luminance

Luma is a weighted sum of specified nonlinear RGB components. Luminance is a photometric quantity obtained from linear values with the appropriate primaries and scale. Weighting and nonlinear encoding generally do not commute. Neither constant luma nor constant luminance guarantees equal perceived brightness across colors.

💡 Big lesson (L4.6) — additive vs multiplicative → choice of encoding

Use linear light for an operation that adds light. Use ratios or log differences when the model concerns multiplicative changes. Choose a perceptual encoding for storage precision or an edit defined in that representation. Log sharpening can be a legitimate contrast edit; it simply implements a different operator from sharpening linear light.

💡 Big lesson (L4.7) — averaging colors belongs in linear light

To average light, decode, combine linear values, and encode for the output interface. For nonnegative weights summing to one and a convex decoding curve, averaging encoded values produces no more light than averaging the decoded inputs. Kernels with negative weights need separate analysis. For alpha compositing, linearize color and use the correct premultiplied-alpha convention; alpha itself is not gamma-encoded color.