• **Vectors and what they mean here.** A length-$n$ list of numbers; geometrically a point/arrow in $n$-D space. **An image *is* a vector** — stack the pixels into one long column (fig-image-as-vector): a megapixel image is a point in a million-dimensional space. Everything we do to images is geometry in that space.

• **Inner product, norm, angle.** $\langle u,v\rangle=u^\top v$ measures alignment; $\|v\|=\sqrt{\langle v,v\rangle}$ is length; orthogonal means $\langle u,v\rangle=0$. The inner product is how a cone "reads" a spectrum, how a filter responds to a patch, how we measure image similarity.

• **Matrices as linear maps.** $y=Ax$ — a matrix sends vectors to vectors, linearly (preserves sums and scaling). Visualize a 2×2 $A$ as deforming the unit square / grid into a parallelogram (fig-matrix-as-transform). A blur, a color-space conversion, a resampling, a perspective warp — all are (or linearize to) $y=Ax$.

• **Linear systems and inverse problems.** Imaging forward models read $y=Ax$ (scene $x$ → measurement $y$); recovering $x$ is solving the system / inverting $A$. Usually $A$ is huge, rank-deficient or noisy, so we don't literally invert — we optimize (next sections). Square vs over/under-determined; rank; null space (what the measurement *cannot* see).

• **Bases, projection, orthogonality.** A basis is a coordinate system; changing basis re-expresses the same vector. Projecting $v$ onto $u$: $\hat v=\frac{\langle u,v\rangle}{\langle u,u\rangle}u$ (fig-vector-projection). In an **orthonormal** basis $\{e_i\}$ the coordinates are just inner products, $x=\sum_i\langle e_i,x\rangle e_i$ — clean, and energy-preserving (Parseval). This is why we love orthogonal bases (Fourier, orthonormal wavelets); **non-orthogonal, non-negative** bases (the cones/primaries of [[Measuring and encoding color|Color technology]]) need a *dual* basis and are the source of color's trickiness.

• **Eigenvectors and the SVD.** Eigenvectors are directions a matrix only scales; for symmetric matrices they're orthogonal (the spectral theorem) — diagonalizing = finding the basis where the operator is just per-axis scaling. The **SVD** $A=U\Sigma V^\top$ generalizes this to any matrix: rotate ($V^\top$), scale ($\Sigma$), rotate ($U$) — the unit circle becomes an ellipse whose axes are the singular values (fig-svd-geometry). It underlies low-rank approximation, PCA, the pseudo-inverse, and the conditioning of deblurring (small singular values → noise blow-up).

• **Tie to the book's recurring move:** convolution is a linear operator that is *diagonalized by the Fourier basis* ([[Linearity, Fourier, Aliasing and deblurring]]); PCA diagonalizes a covariance; pyramids/wavelets are change-of-basis. Same idea each time — pick the basis where the operator is simple.

• **Derivative = slope = rate of change.** $f'(x)$ is the tangent slope (fig-derivative-gradient, left). In images, the **discrete derivative** (finite difference, neighbor minus neighbor) is large at **edges** — the basis of Sobel gradients and sharpening ([[Convolution]]).

• **Partial derivatives and the gradient.** For $f(x_1,\dots,x_n)$, the gradient $\nabla f$ stacks the partials; it points in the direction of **steepest ascent**, and its magnitude is the steepness (fig-derivative-gradient, right: arrows on a contour map). Edge *orientation* is the direction of the image gradient.

• **Jacobian and Hessian — the multivariable kit.** A map that takes a vector and returns a vector, $\mathbf f:\mathbb R^n\to\mathbb R^m$, has a **Jacobian** $J$: the $m\times n$ matrix of all first partials $J_{ij}=\partial f_i/\partial x_j$ — the best **linear approximation** of $\mathbf f$ near a point (the gradient is just the scalar, $m{=}1$, case), and composing maps **multiplies** their Jacobians (the vector chain rule that backprop runs on; also how a warp is linearized locally). A **scalar** $f$ has a **Hessian** $H$: the $n\times n$ matrix of *second* partials $H_{ij}=\partial^2 f/\partial x_i\partial x_j$, i.e. **curvature**. At a stationary point ($\nabla f=0$) a **positive-definite** Hessian certifies a genuine **minimum** (vs a saddle), and **Newton's method** steps by $H^{-1}\nabla f$ — second-order optimization (Gauss–Newton for least squares). Keep it conceptual; the Jacobian is the one you meet constantly.

• **Chain rule.** Derivative of a composition — the engine of **backpropagation**: a deep network is a long composition, and training differentiates the loss through it by repeated chain rule. State it once here so the ML section can lean on it.

• **Integrals = accumulation.** Area under a curve; in imaging a **pixel value is an integral** — radiance integrated over the sensor's area, solid angle, exposure time, and spectral response ([[Image formation and linear perspective]]). Continuous vs discrete (sums) — we constantly move between them (sampling, [[Resampling and upsampling|Resampling]]).

• **Convexity, minima, and why gradients matter.** A function is minimized where $\nabla f=0$; if it's **convex** (bowl-shaped) that's the global minimum. This is the bridge to optimization: follow $-\nabla f$ downhill.

• **The pattern: loss = data term + prior.** Almost every recovery problem is $\min_x \underbrace{\|Ax-y\|^2}_{\text{fit the data}}+\lambda\, \underbrace{R(x)}_{\text{prefer plausible }x}$. Denoise, deblur, demosaic, HDR-merge, white-balance — same skeleton, different $A$ and $R$.

• **Least squares & the normal equations.** Fitting a line/plane/model by minimizing squared residuals (fig-least-squares: points, fit, vertical residuals); closed form $A^\top A\hat x=A^\top y$ (and the SVD/pseudo-inverse when $A^\top A$ is singular). Why *squared* error — Gaussian-noise MLE (ties to the probability section), and it's the one that makes the math linear; its blindness to outliers motivates robust losses.

• **Gradient descent.** When there's no closed form (huge or nonlinear problems), step downhill: $x_{t+1}=x_t-\eta\nabla f$ (fig-gradient-descent: descent path on a contour bowl). Learning rate $\eta$ — too big overshoots/diverges, too small crawls. Variants in one breath: stochastic GD (a minibatch per step — how nets train), momentum/Adam. Convex → one basin; non-convex (deep nets) → local minima, and yet it works.

• **Ill-posedness & regularization.** Inverse problems often have many $x$ explaining $y$ (or noise blows up the inverse). A **regularizer** $R(x)$ encodes a prior — smoothness (Tikhonov), sparsity ($\ell_1$, total variation), or a learned prior. $\lambda$ trades data-fit against prior. This is the principled cure for the deblurring noise-amplification of [[Linearity, Fourier, Aliasing and deblurring]].

• **Overfitting and the bias–variance trade-off.** Fit the noise and you generalize badly (fig-overfitting: under-/just-right/over-fit polynomials). Capacity, regularization, and held-out validation — the same dial as $\lambda$, and the bridge to the ML section.

• **the basics, in one breath.** A **probability** is a number in $[0,1]$; **independent** events **multiply** (AND), **mutually-exclusive** ones **add** (OR). A **distribution** says how probability is spread over the possible outcomes; its **expectation** $E[X]$ is the long-run average and its **variance** the spread. That is the whole vocabulary the rest of this section leans on — kept deliberately simple.

• **Random variables, distributions, expectation, variance.** A pixel reading is a random variable around the true value; mean = the value we want, **variance/std = the noise**. Expectation is linear; variances of independent things add.

• **The two distributions we actually use.** **Gaussian** $\mathcal N(\mu,\sigma^2)$ — the bell curve (fig-gaussian: μ, ±σ), read noise, the central limit theorem, and the reason squared error is "natural." **Poisson** — photon counting: shot noise with $\mathrm{variance}=\mathrm{mean}$, so std $=\sqrt N$ → SNR is **worse in the shadows** ([[Image formation and linear perspective]]).

• **Independence and averaging.** Average $N$ independent noisy measurements → variance drops by $N$, std by $\sqrt N$. This single fact is why we stack frames, why bigger pixels are cleaner, and the whole "quality through quantity" thread.

• **Conditional probability and Bayes.** $p(x\mid y)\propto p(y\mid x)\,p(x)$ (fig-bayes-posterior: prior × likelihood → posterior). **MAP estimation** = maximize the posterior = the optimization "data term ($\log$ likelihood) + prior ($\log$ prior)" of the last section — the two views are the same. Discounting the illuminant (white balance), denoising, and demosaicking are all Bayesian estimates.

• **Information & entropy.** $H=-\sum p\log p$ = average surprise = the bits needed to encode a source (fig-entropy-coin: $H(p)$ peaks at $p=0.5$). Low entropy = predictable = compressible — the theory behind entropy coding in [[File formats and compression]]. Mutual information as "shared bits." Keep it light: enough to know *why* JPEG can shrink an image and *why* it can't shrink random noise.

• **Supervised learning = curve fitting, scaled up.** Given pairs $(x_i,y_i)$, find $f_\theta$ minimizing average loss $\mathcal L(\theta)$ — exactly the regression of two sections ago, with millions of parameters. Train/validation/test split; generalization is the goal, not memorization (callback to fig-overfitting).

• **The artificial neuron and layers.** A neuron computes $\sigma(w^\top x+b)$ — a weighted sum (an inner product!) through a nonlinearity (fig-neuron). Stack them into layers → a **multilayer perceptron**; a deep net is a long composition $f_\theta=f_L\circ\dots\circ f_1$ of such building blocks, hence very expressive.

• **Training = gradient descent + backprop.** Minimize $\mathcal L(\theta)$ by SGD; **backpropagation** is the chain rule (calculus section) computing $\nabla_\theta\mathcal L$ efficiently through the composition. Autodiff frameworks (PyTorch) do this for you (programming section).

• **Convolutional networks — the imaging-native architecture.** Weight-shared, local, shift-equivariant filters — literally learned convolutions ([[Convolution]]), so they fit images naturally; encoder–decoder / U-Net shapes for image-to-image tasks (denoise, demosaic, super-res). Brief, with forward-ref to the advanced books for depth.

• **What changes vs classical methods.** Same "loss = data-fit + prior" skeleton; the **prior is now learned** from a dataset rather than hand-designed. Strengths (state-of-the-art quality), costs (data, compute, failure modes, hallucinated detail — tie to the ethics discussion in [[Human factors and the art of photography]]).

• **Why two languages.** The book renders as a **Python book** *or* a **C++ book** from one source ([[Book Generation Process]]); this section is the shared mental model. Python/NumPy: arrays, vectorization, broadcasting — read like the math. C++: the same operations when speed/memory matters (ISP, Halide).

• **Images as arrays/tensors.** An image is an $H\times W\times C$ array (NumPy ndarray / `std::vector` / `torch.Tensor`); operations are whole-array (vectorized) not per-pixel loops — the linear-algebra view made executable. Indexing, slicing, broadcasting; row-major memory and why loop order matters (forward-ref to [[Making computational photography code fast]]).

• **The speed differential.** For a low-level inner loop you write yourself (e.g. your own convolution), expect roughly **~1000× Python→C++** and **~10× more again for an optimized [[Making computational photography code fast|Halide]] schedule** — why hot loops leave Python; see [[Making computational photography code fast]].

• **PyTorch in one paragraph.** Tensors that live on CPU or GPU, plus **autograd** — every operation records a graph so `loss.backward()` gives gradients for free (the backprop of the ML section). This is why modern imaging code is written in it even when no "learning" is involved: free derivatives + GPU.

• **Numerical hygiene.** Floats vs ints, value ranges and clipping, **linear vs gamma space** before arithmetic (the encoding Big Lesson), in-place vs copy, NaNs/inf from divide-by-zero — the bugs that actually bite, cross-ref [[Developing, Testing and Debugging]].

• **Pointers to practice.** The dual-language listings, the `imageops`/`figstyle` style, and [[Exercises & Experiments]] warmups (load an image → it's a NumPy array → do arithmetic → save) that make all of the above concrete.

• The central design decision: keep two living documents apart.

• **Outline** = an annotated spec of *what* the book says — structure, order of ideas, and per-section figures, equations/symbols, prerequisites, and the papers/slides it draws on.

• **Prose** = the actual text a reader sees — about *how* it reads. A **compilation** step turns outline (plus supporting files) into draft prose, the way a compiler turns source into a program.

• Point of the separation: leverage — structure/content reasoned about and rearranged cheaply in one place, voice/phrasing in another.

• The arrow runs both ways: when the author edits prose, the change is **triaged** home — *content* (new point, reorder, corrected claim) → outline; *voice* (word choice, rhythm) → style guide; purely local polish → locked in place so it is never silently overwritten.

• Central promise: *re-compiling reproduces essentially the author's edits* rather than regenerating something new. The outline is kept a strict **superset** of the prose's content, so nothing emerged-in-writing is lost on the next pass.

• LLMs are at their worst in long, drifting conversations where early mistakes calcify and context "rots." Antidote: **files are the durable memory, the chat is disposable.**

• Each section is compiled in a **fresh, clean context** from an explicit **capsule** of exactly what it needs — never from accumulated conversation.

• The capsule holds: the section's outline bullets; a one-line chapter thesis + neighboring sections for continuity; in-scope notation and relevant style rules; and — the key part — the **source material**.

• Two kinds of source, different roles: (1) the teaching the book grew from — lecture **slides** and cleaned **transcripts**, as machine-readable digests — supply *pedagogy and voice* (framing, worked examples, analogies, order of reveal); (2) the technical literature — cited **papers and textbooks** on disk — supply *precision* (definitions, results, attributions).

• Division of labor: slides/transcripts decide *how* to explain, papers decide *what is true*. Grounding every claim in a named source is the first defense against fabrication. The rule: write from the sources, not the model's memory.

• No diagram or example image is drawn by hand. Each figure is a small **program** that builds it — regenerates from source, stays consistent with the others, updates when data/style changes (the same reproducibility ethic the book preaches for image-processing code).

• Two libraries: `imageops` holds the actual image **operations** a figure demonstrates (exposure, contrast, tone mapping, convolution, pyramids, demosaicking…), written once so figure and prose can never disagree; `figstyle` holds one shared **visual style** — one palette, one font set, consistent labels — applied everywhere.

• A figure is specified in the outline alongside the text it supports, built to a **legibility standard** (no text smaller than body type, nothing clipped/overlapping), and checked so the prose actually *leverages* it.

• Photographs used as examples are the author's own, or carefully credited Creative-Commons images.

• A pleasing bit of recursion: the book's **decorative** imagery was made with a **generative text-to-image model** — the same diffusion/transformer family the book explains in [[Generative AI and diffusion]]. A book about computational photography, illustrated by computational photography's newest tool.

• The line to keep straight: **technical figures are code** (previous section) — generative models are *not* trusted to draw a ray diagram or a Bayer grid, where a hallucinated arrow or a mangled equation would be a lie. The model is used only where invention is the point: **ornament**, not evidence.

• **The model.** We used **Reve**, a text-to-image API, driven by two small scripts that read a key, post a prompt (or an image-edit instruction), and save the returned PNG — each call metered in credits, so generation is always a deliberate, bounded selection, never an open faucet.

• **Cover and title art** (`reve_gen.py` ← `cali.txt`). A bank of prompts, each pairing the book's title with one *calligraphic hand* (copperplate, blackletter, Spencerian, Carolingian, Insular/Celtic, Art Nouveau, Roman capitals, brush) and one *gloriously over-engineered futuristic camera* bristling with mismatched lenses — rendered as paper-quilling, gold leaf, illuminated manuscript, and so on. Each prompt is generated in **several variants**; the author picks the keepers. The whole cover suite is one prompt file plus one command.

• **Stylized figures** (`reve_stylize.py` ← `figstyles.txt`). The *opposite* direction — **image-to-image**: take an already-built, typeset figure and ask Reve to **re-render it in another artistic medium** (chalk on a blackboard, blueprint, da-Vinci ink, Diderot engraving, watercolor…) while *preserving every label and equation*. Used for chapter-opener flavor, never to compute or alter content.

• **The fidelity guard.** Generative edits love to redraw a typeset "$r_d$" as literal LaTeX or drop a subscript. Two defences: every edit instruction carries an explicit *keep the math as real typeset glyphs, no backslashes/dollar-signs* clause, and each result is optionally checked by a **vision model** (`reve_verify.py`) that compares it against the original and flags any **major** content drift (a missing panel, a changed sign, an illegible label) — cosmetic medium differences pass, meaning-changing ones are rejected.

• **Honesty.** Every generated image is decorative or illustrative and labeled as such; none is presented as a photograph, a dataset sample, or the output of an algorithm. Where a figure *mimics* a result without being one, it carries a visible **SIMULATED** mark (see the figure conventions). Provenance — model, prompt, and that the image is synthetic — is recorded with the asset.

• A second generative tool, in a different medium: where a figure or an **interactive demo** needs a **3D model** — an insect, a camera, an eyeball, a studio umbrella, a human figure — we generate it from a text prompt with **Meshy**, a **text-to-3D** API, rather than hunting for a correctly-licensed mesh. The same recursion as the cover art: a book about imaging, populated by models a generative system imagined.

• **Where it is used.** The book's **interactive WebGL demos** for Photography 101 (the exposure triangle, portrait lighting, dolly zoom, depth of field, and face-distortion simulators) need real 3D geometry to light, focus, and reproject: a beetle / bee / ladybug for the macro depth-of-field bokeh, flowers for the blurred background, studio umbrellas for the lighting rig, diverse human figures for the portrait and perspective demos, and props (cameras, an eye) for general illustration.

• **The pipeline** (`generate_mesh.py` ← a prompt). One small script reads a key, posts a prompt, polls until the model is ready, downloads a **GLB**, and embeds it as base64 so it loads offline — no server, even from a local `file://` page. Two modes mirror a credit/quality trade-off: **preview** returns *untextured* geometry (cheaper) for demos that apply their **own** material — a metallic shell, a grid — and **refine** adds **PBR textures** when color carries the meaning (a bee's stripes, a ladybug's spots). Untextured meshes are decimated to a light polygon budget; textured ones are kept whole (decimation would drop the texture coordinates). Each call is metered in credits, so generation stays a deliberate, bounded selection, never an open faucet.

• **Diversity, on purpose.** The human figures are generated as a deliberately **diverse set** — varied gender, race, age, and dress — so the people who populate the demos and figures are not a monoculture.

• **Honesty and credit.** As with the text-to-image art, these models are **illustrative stand-ins**, never presented as photographs of real objects, specimens, or people, and their synthetic provenance is recorded with the asset. **Every caption that uses a Meshy-generated model acknowledges it** — e.g. *"3D model generated with Meshy AI"* — the convention is firm: AI-made assets are labeled where they appear.

• A third generative medium: the cover art is also brought to life as short, silent, **seamlessly-looping videos** for the web edition — the index page offers an **animated / static** toggle, and animated mode plays one of the loops at random (`cover_animate.py` ← a cover image, via **Kling** image-to-video on **fal.ai**).

• **The hard part is the loop.** The obvious way to force a true loop — pin the model's *end frame* to the *start frame* — backfires: the laziest path to landing on the identical final frame is to barely move, so the clip comes out nearly frozen. The loops are made the other way round: generate slow, calm, **cyclic** motion — prompted so every element *returns to its starting state* (a full 360° rotation, figures walking back to their pose) — then close the seam with a short **forward crossfade-wrap** (a brief dissolve, never a reversing "ping-pong"). A flawless seam with lively motion is genuinely hard; truly cyclic motion, or a purpose-built looping-context video model, is the clean route.

• **Honesty.** Like the still art, the animations are generative and decorative — never presented as captured footage; their synthetic provenance is recorded with the asset.

• A chapter is not allowed to start drafting until it passes a **readiness gate**: outline fleshed out; prerequisites declared and satisfied upstream; figures built (or stub-placeheld); equations/notation locked; references resolving to real documents; source coverage checked.

• After a draft exists, it is run past a panel of automated **reviewers**, each a different lens: technical (inaccuracies, missing assumptions); pedagogical (where a student gets lost); prerequisite (assumed-but-never-introduced background); style (single-author voice); plus figure–text fit, citation correctness, and faithful coverage of slides/papers.

• Findings collect into a running issues list and steer the next revision. Mechanical checks — consistent notation, language-isolated code, sane value-range/encoding assumptions — run separately and automatically.

• **A second opinion from a rival model.** Because the drafting AI shares blind spots with its own prose, an *independent* model from a different vendor (here, OpenAI's GPT) is run as an outside reviewer over the outline or the drafts — at chapter, part, or whole-book scope — tasked only with disagreeing. Its notes are triaged: small, unambiguous fixes the editor applies directly; judgment calls escalated to the author. Crossing model families catches errors a single model would wave through (it flagged, for instance, a real optics slip in an early draft).

• The verifiers don't replace human review; they make it land on what matters.

• Hardest problem in piecewise generation: the pieces must add up to one object. Three measures.

• **Single-source registries**: one master **notation** table owns every symbol; one **glossary** owns every key term; a registry of recurring **"big lessons"** (imaging is multiplicative; color is non-orthogonal and non-negative; the right basis diagonalizes; edge-preserving filtering is about affinity; quantization is rarely the culprit) records where each first appears and recurs. Defined in one place, referenced everywhere → cannot drift.

• **Dependency graph**: each chapter declares what it *provides* and *requires*; the build checks nothing is used before introduced, and propagates a changed shared definition to every dependent chapter.

• **Version control**: every change to prose and tooling is committed, so generated and edited versions sit side by side, edits extract as diffs, and history is recoverable. A verbatim **log of every instruction** and a dated **process changelog** accompany the repo, so the (continuously evolving) method can be reconstructed — much of what this appendix is built from.

• Prose written once in **neutral markup**, rendered to several outputs — print/PDF and a web edition — from a single source, so it is proofread once, not per-format.

• The same single-source trick produces two flavors, one with **Python** code and one with **C++**: shared explanation written once, language-specific snippets swapped in per build, so a reader sees only one language, never both.

• Derived artifacts — slide digests, figures, and eventually compiled chapters, index, glossary — follow an **incremental, build-style** discipline: each records its inputs and rebuilds only when those change, so a small edit doesn't trigger wholesale regeneration.

• The book carries a **version number** that ticks up on each recompile.

• "Written by AI" is both an overstatement and an undersell — worth being precise about the division of labor.

• The AI was excellent at the laborious-but-well-specified: drafting fluent prose from a tight, source-backed outline; writing and debugging figure-generation code; mechanical bookkeeping (notation tables, cross-references, consistency sweeps); and building the tooling that ran the process. It gave one author the throughput of a small team.

• It was *not* trusted with judgment. A human chose structure and argument, decided what mattered and what to cut, supplied the taste, caught subtle errors that survive plausible prose, and owns every remaining claim.

• The safeguards above — source-grounding vs fabrication, fresh per-section context vs drift, single-source registries vs incoherence, a reviewer panel feeding a human editor — exist precisely because an unsupervised model would fail in each way.

• Honest summary: the AI is a powerful collaborator and tireless build engine; the responsibility, like the byline, stays human.

• an honest, prompt-log-reconstructed table of the **human vs AI** split, **per part**, for the **outline** (content/structure) and the **figures** — rounded to 5%, proportions not precision.

• caveat up front: "authorship" of the outline is fuzzy — the human set structure, chose every topic, and supplied the course material, while the AI expanded those seeds into connective text; a low human % means *directed-not-absent*. A figure counted "AI-originated" was still requested in general terms and accepted/killed by the human.

• two metrics: **outline human %** (originated as explicit human direction / course material vs AI expansion) and **figures human-specified %** (named/described by the human vs AI-proposed from a "suggest the figures" prompt).

• patterns: human outline-share peaks where the book leans on the author's course/research (Fundamentals, Basic, Optics, Multiple-exposure, Performance/Halide, pset Appendices) and dips in survey-style later parts; human figure-share concentrates in the drafted-and-iterated parts (Fundamentals, Basic) and the Appendices (author's own handout figures). Whole-book ≈ **30–35%** human on the outline, ≈ **20%** human-specified on the figures — everything reviewed and owned by the human regardless of who first drafted it.

• The inversion: drafts are the source, the outline is the spec for what is not yet written.

• What was lost with recompilation, and what replaced it (version control, stable ids, gates).

• The loop: read the rendered page, say what should be different, an agent makes the change.

• Batches run to completion; audits rather than checkpoints; corrections are the strongest signal.

• What an agent decides alone, what it must ask, and the standing rule about deploying.

• The roster and what each owns; report-do-not-fix; a task that is not yours.

• The shared contract, the state file, the inbox, the objections file.

• What a skill must state; the ones that carry the most weight.

• Twenty-six numbered checks in the publish script; a gate rather than a rule.

• Coordination, not fabrication: concurrent edits, lost handoffs, builds racing each other.

• Measured tokens and words, and what the numbers do and do not mean.

• What is in the downloadable archive and how to read it.