BookNotesSlidesP-sets Syllabusdraft · v0.2.49.1
◀▲⌂▶
💬Comments welcome. To leave a note, select any text and click the note / highlight button that pops up — or open the panel with the tab at the top-right (‹). Notes are visible only inside our private review group.

45.15 Rendering a raw DNG⧉

A raw DNG is not a picture. It is a grid of sensor measurements plus a recipe for turning them into one (this is the thesis of Chapter 45.14, DNG, the Digital Negative). "Render it" hides two different jobs that this book keeps deliberately separate:

  1. The radiometric job. Recover a scene-linear image: values proportional to the light that fell on the sensor, white-balanced and in a known color space (we use linear sRGB primaries). This is useful input for algorithms whose forward models assume linear light, including HDR merging and optical blur. Demosaicking itself starts from mosaic samples, and some denoising or editing methods deliberately use other representations (see Chapter 8.1, Image representation).
  2. The cosmetic job. Produce a pleasing displayed image that matches what the photographer saw in Lightroom. Interpreting linear numbers directly as sRGB makes them too dark. Even after correct display encoding, a neutral render may look flatter than the photographer's Lightroom result, which also includes a baseline tone curve that lifts the midtones, a gentle highlight roll-off, and a touch of saturation. We call that final step the beauty pass.
💡 The one rule

Do all processing in scene-linear; apply the beauty pass only at the very end, for display. Never blend, blur, merge, or solve on the beautified image, and never bake the beauty pass into a file you will later compute on. The beauty pass is a display transform, not data.

45.15.1 The pipeline, step by step⧉

Inputs: the original Bayer DNG (libraw/rawpy can read it) and its embedded XMP develop settings written by Lightroom. Our reference implementation lives in dng_linear_correct.py (the radiometric job, baked once into a demosaicked LinearRaw DNG) and figures/imageops.py (load_linear_dng, then linear_to_display, the beauty pass).

1. Read the Lightroom develop settings (XMP). From the DNG's XMP packet, read the crs: fields: Exposure2012 (in stops), Temperature and Tint (white balance), WhiteBalance (As Shot / Custom), and the crop rectangle (HasCrop, CropTop/Left/Bottom/Right, CropAngle). These are the photographer's artistic decisions; honoring them is what makes the render match Lightroom rather than some neutral default.

2. Demosaic to camera-native linear. Demosaic the Bayer mosaic with no gamma, no auto-brighten, and no white balance yet: rawpy.postprocess(gamma=(1,1), no_auto_bright=True, user_wb=[1,1,1,1], output_color=raw, output_bps=16). Here raw means rawpy.ColorSpace.raw. The result is a 16-bit camera-native RGB rendering with linear encoding. Decoder normalization, clipping, and demosaicking have already affected the samples; it is not untouched sensor data.

3. White balance from Temperature/Tint. Convert Lightroom's Temperature/Tint to camera-channel multipliers using Adobe's own dng_temperature algorithm (the kTempTable/kTintScale from the DNG SDK: temperature→$xy$ chromaticity, then through the DNG ColorMatrix/CameraCalibration to camera-neutral multipliers), and multiply the camera RGB by them (green normalized to 1). Using Temperature/Tint (not the camera's as-shot neutral) is what reproduces a custom white balance set in Lightroom.

4. Color: camera-native linear → linear sRGB. The current _color_matrix_to_srgb helper first forms B = ColorMatrix @ sRGB_to_XYZ, divides each row of B by its row sum, and inverts the result. The row normalization makes a neutral input map to output white. It prefers ColorMatrix2 and uses ColorMatrix1 if necessary. This is the implemented approximation, not a complete implementation of every DNG color-profile option.

The earlier path, retained as a fallback when ColorMatrix is absent, uses ForwardMatrix (reference camera coordinates → XYZ D50) and CameraCalibration (CC). Its composition is

$$ \text{cameraToXYZ}_{D50} \;=\; \mathrm{FM}\cdot \operatorname{diag}\!\big(1/\text{refNeutral}\big)\cdot (\mathrm{AB}\cdot \mathrm{CC})^{-1}, \qquad \text{refNeutral} = (\mathrm{AB}\cdot \mathrm{CC})^{-1}\cdot \text{AsShotNeutral}, $$

with AB the AnalogBalance (identity for us). Then $\text{linear sRGB} = M_{\text{XYZ(D50)}\to\text{sRGB}}\cdot \text{cameraToXYZ}_{D50}\cdot \text{cameraRGB}$. The reference-neutral normalization and inverse calibration belong together. In the intermediate files, white balance is already baked in and AsShotNeutral is written as $[1,1,1]$; with identity AB, refNeutral is then $\mathrm{CC}^{-1}\mathbf{1}$. For diagonal CC, those calibration factors cancel in this composition. Do not apply the original white-balance gains a second time. A general decoder must also honor the profile's calibration conventions and illuminant interpolation.

5. Exposure. Multiply the linear image by $2^{\,\text{Exposure2012}}$. In linear light, exposure is a plain multiply (see Chapter 8.3, Tone point operations).

6. Crop (and straighten). Apply the crop rectangle and any straightening angle.

The stored intermediate is a processed camera-RGB image with white balance, exposure, and crop baked in; the loader then converts it to linear-sRGB primaries. The implementation clips values to $[0,1]$, and its optional max_px path resizes through an 8-bit encoded image before decoding again. Thus output="linear" describes the returned encoding, not calibrated ground truth or lossless preservation of all sensor information. These distinctions matter when interpreting quantitative demonstrations.

Reference implementation (steps 1–6, pseudocode)
s     = read_xmp(dng)                         # crs:Exposure2012, Temperature, Tint, Crop*
cam   = rawpy.postprocess(dng,               # demosaic only — camera-native LINEAR
          output_color=raw, gamma=(1,1), no_auto_bright=True,
          user_wb=[1,1,1,1], output_bps=16) / white_level     # → [0,1]
mult  = wb_multipliers(s.Temperature, s.Tint, color_tags)     # Adobe dng_temperature
cam  *= mult                                  # white balance (green-normalized)
CC, FM, AB = color_tags.CameraCalibration2, ForwardMatrix2, AnalogBalance(or I)
refN  = inv(AB·CC) · AsShotNeutral            # AsShotNeutral now [1,1,1] (WB baked)
M     = XYZ_D50_to_sRGB · FM · diag(1/refN) · inv(AB·CC)
lin   = clip(cam · Mᵀ, 0, 1)                  # linear sRGB primaries
lin  *= 2**s.Exposure2012                     # exposure (a multiply in linear)
lin   = crop(lin, s.Crop*)                    # crop + straighten
# `lin` is the scene-linear ground truth. Beauty pass (step 7) is display-only.

(The two reference 3×3s: XYZ_D50_to_sRGB is the standard Bradford-adapted matrix; FM/CC come from the DNG tags, interpolated between the two calibration illuminants by correlated color temperature; we use the higher-CCT pair, index 2, e.g. D65.)

7. The beauty pass (display only). To display, encode to sRGB and apply the camera/Lightroom look: a fitted baseline tone curve, whose midtone lift depends on the chosen look and input exposure, a highlight roll-off, and modest saturation. Our linear_to_display uses a smooth, monotone baseline curve, a logistic S about a pivot, settled against twenty-two of our own photographs at once (an earlier version was fitted to Lightroom's luminance distribution, which worked but carried more parameters than the shape needs; see Chapter 8.4, Beauty curves: the camera "look"). It applies that curve with a highlight-weighted hybrid roll-off: midtones are toned hue-preserving so saturated colors stay saturated, but as luminance approaches white the curve goes per-channel so bright skies and the sun desaturate cleanly to white instead of taking on a magenta/cyan cast. The color half is a small vibrance rather than a flat saturation: chroma is scaled in Oklab, by a gain that fades as the pixel's own chroma rises and is notched down on skin hues, and it eases off in the same highlights the roll-off is whitening. How colorful a pixel already is, and whether its hue is skin, are both measured in Oklab rather than as HSV saturation and hue on the encoded value, because HSV saturation ignores lightness and so cannot tell a dark plum from a bright pink.

fig-dng-beauty-pass
Figure 45.15.1. The same scene-linear DNG render, before and after the beauty pass, on three frames that give the pass three different jobs: a saturated subject against foliage, a hillside that is nearly all one hue, and a dark interior where the lift is the whole story. Left of each pair: the radiometrically-correct scene-linear image, sRGB-encoded, dull and dark, because it carries no camera tone curve. Right: after the beauty pass (the midtone-lifting curve, the highlight roll-off, and a small vibrance), which is what a viewer expects. The pixels feeding any algorithm are the left image; the right is display only.

45.15.2 Where Lightroom's "look" actually comes from⧉

Lightroom's render is not just the color matrix. With an "Adobe Standard" or "Adobe Color" DNG camera profile (DCP), the chain is: matrix → HueSatMap (an HSV calibration table) → LookTable (the creative "look") → the baseline tone curve. Our pipeline uses the matrix approximation described above and approximates a display look with the beauty pass, but it does not apply the profile's HueSatMap/LookTable. In testing, the matrix-plus-beauty render matches the Lightroom JPG closely; applying our own re-implementation of the HueSatMap/LookTable helped only marginally and risked worse color (a too-purple blue), because the LookTable's value-encoding and HSV conventions are easy to get subtly wrong. So we deliberately stop at matrix + beauty. A close visual match on these examples does not establish a complete Camera Raw reproduction. Profile tables, illuminant interpolation, and process-version behavior can matter on other photographs.

45.15.3 What we would ask an Adobe engineer to check⧉

The implementation raises useful questions for validating any independent raw developer. The earlier ForwardMatrix investigation is retained here because it explains the calibration factors, even though the current default uses ColorMatrix:

  1. The forward transform. Is cameraToXYZ_D50 = FM · diag(1/refNeutral) · (AB·CC)⁻¹, with refNeutral = (AB·CC)⁻¹ · AsShotNeutral, the correct composition? Our bug was applying FM · CC (un-inverted, no reference-neutral); the relevant check is whether the full conversion, including white balance and profile selection, matches the intended Camera Raw behavior, and that picking the index-2 (higher-CCT) FM/CC rather than CCT-interpolating between illuminants 1 and 2 is acceptable for daylight scenes.
  2. Skipping HueSatMap / LookTable. We apply the matrix + a baseline tone curve but not the profile's HueSatMap or LookTable. Is matrix-plus-baseline-curve a defensible approximation of "Adobe Standard"/"Adobe Color," or does omitting those tables introduce errors that matter (and if we did apply them, what are the exact value-encoding and HSV-space conventions for the LookTable)?
  3. The baseline tone curve. We recovered a baseline curve empirically (fitting our linear-render luminance to the Lightroom-JPG luminance). Is there a canonical baseline tone curve for the current Process Version we should use instead, and is it applied per-channel, on luminance, or in some other space?
  4. Highlight roll-off. We desaturate highlights toward white with a luminance-weighted blend toward a per-channel curve. How does Camera Raw actually roll highlights to white?

45.15.4 Two kinds of DNG, and the special cases⧉

45.15.5 Easy mistakes (most of which we made)⧉

This pipeline is a minefield, and a wrong render looks almost right. In rough order of how badly each bit us:

fig-dng-mistakes
Figure 45.15.2. Easy ways to get it wrong, versus the correct pipeline, the same photo throughout: limestone under tungsten, chosen because it is the frame whose two white balances are furthest apart, so the second panel has something to show. Top row, three failures: double gamma (a finished display image encoded a second time, milky and washed); as-shot white balance (ignores the balance the photographer set, so the whole cave runs warm); dropped exposure (beauty pass without baking the Lightroom exposure, so too dark). Bottom row: the incomplete render (correct scene-linear, but no beauty pass, so dull); the correct pipeline (LR exposure/WB/crop, corrected color, beauty pass); and the reference Lightroom JPG. Each failure looks superficially plausible, which is exactly why these bugs survive.