45.18 How this book was created (the early process)🔗⧉

This appendix is frozen. It records the method the book was built with, and that method has since been replaced: the drafts became the source rather than the output, and the single collaborator became eleven agents with written ownership of separate parts of the project. The current account, with the working relationship, the multi-agent internals, the gates and a downloadable bundle of the real files, is Chapter 45.19, How this book is made now. Everything below is left as it stood, including its numbers, because a methods note that is quietly updated is not a record of anything.

This book was written with an AI collaborator, Anthropic's Claude, as a deliberate experiment in AI-assisted authoring. This appendix describes, plainly, how the outline, the figures, the prose, and even the book's own tooling came to be. An AI handled much of the drafting, building, and checking; a human made the decisions. The method matters because it affects accuracy, consistency, and reproducibility, and because the failure modes everyone rightly worries about (hallucinated facts, a soup of inconsistent notation, prose that drifts into nonsense over hundreds of pages) are exactly the things the method was built to prevent.

45.18.1 Two documents, not one🔗⧉

The central design decision was to keep two living documents apart. The first is the outline, an annotated specification of what the book says: its structure, the order of ideas, and, attached to each section, the figures it needs, the equations and symbols it uses, its prerequisites, and the papers and slides it draws on. The second is the prose, the actual text a reader sees, which is about how it reads. A compilation step turns the outline (plus a set of supporting files) into draft prose, the way a compiler turns source into a program. The split keeps structure separate from phrasing: structure and content live in one place where they can be reasoned about and rearranged cheaply, while voice and phrasing live in another.

Edits also flow back from the prose to the outline and style guide. When the human author edits the prose, the change is triaged back into the right home: a change to content (a new point, a reordering, a corrected claim) updates the outline; a change to voice (word choice, rhythm, "spell this out") updates a style guide; and a purely local polish is simply locked in place so it is never silently overwritten. This split makes one promise possible: that re-compiling the book reproduces essentially the edits the author made, rather than regenerating something new and forgetting the corrections. The outline is therefore kept as a strict superset of the prose's content, so nothing the writing made emerge is lost on the next pass.

A note about how this worked in practice, because the plan and the practice diverged. The intention was that nearly all of the human editing would happen in the outline files: that I would shape structure and content upstream and let prose simply fall out of recompilation, rarely touching the drafts directly. That held for the initial pass, while the outline was being built and the book's skeleton was still forming. But once real draft prose existed, the natural thing to do was to read the rendered HTML drafts (the book as a reader would actually meet it) and react to that: spotting a clumsy transition, a missing figure, a claim that needed nuance, an interactive that misbehaved. So most of the later editing happened not by hand-editing files at all, but by describing the change in a prompt to the AI collaborator, which then made the edit and triaged it to the right home (outline, style guide, or a locked-in local fix) by the rules above. The two-document discipline survived; what changed was the interface to it: from "edit the specification" to "read the result and say what should be different," with the machine maintaining the specification underneath. The durable artifact stayed structured and single-sourced, while the human worked through the most convenient surface available.

45.18.2 Compiling a section🔗⧉

Large language models are at their worst in long, meandering conversations, where early mistakes calcify and context "rots." The system keeps durable state in files, not in chat history. Each section is compiled in a fresh, clean context from an explicit capsule of exactly what it needs, never from accumulated conversation. That capsule holds the section's outline bullets, a one-line statement of the chapter's thesis and the neighboring sections for continuity, the in-scope notation and the relevant style rules, and, most importantly, the source material.

Two kinds of source feed every section, and they play different roles. The first is the body of teaching this book grew out of: the lecture slides and cleaned transcripts from the course, transcribed into machine-readable digests. These supply pedagogy and voice: the specific framing, the worked examples, the analogies, and the order in which a human teacher found it natural to reveal things. The second is the technical literature: the cited papers and textbooks, kept on disk and consulted for the exact definitions, results, and attributions. These supply precision. The division of labor is deliberate (slides and transcripts decide how to explain, papers decide what is true), and grounding every claim in a named source is the first and best defense against an AI confidently inventing things. The rule throughout is to write from the sources, not from the model's own memory.

45.18.3 Figures as code🔗⧉

None of the book's diagrams or example images is drawn by hand. Each figure is a small program that builds it, so that every figure regenerates from source, stays consistent with the others, and updates automatically when its data or style changes: the same reproducibility rule used throughout the book for image-processing code. Two little libraries do the work: one holds the actual image operations a figure demonstrates (exposure, contrast, tone mapping, convolution, pyramids, demosaicking, and so on), written once so a figure and the book's prose can never disagree about what an operation does; the other holds a single shared visual style (one palette, one set of fonts, consistent labels) applied everywhere. A figure is specified in the outline alongside the text it supports, built to a legibility standard (no text smaller than the body type, nothing clipped or overlapping), and checked against the prose so that the writing actually uses each figure rather than merely gesturing at it. Photographs used as examples are the author's own, or carefully credited Creative-Commons images.

45.18.4 Generative imagery: cover art and 3D🔗⧉

The line the previous section draws (figures are code, never guessed) is exactly what lets the book use generative models freely for the decorative imagery, where invention is the point rather than evidence. The cover art was made with a text-to-image model (Reve), and the book's 3D assets (the fantastical cameras and optical artifacts that populate the interactive demos and the animated cover) were made from text prompts with Meshy, a text-to-3D tool, then placed and lit in a small three.js scene. The rule stays firm: generative models are never trusted to draw a ray diagram or a Bayer grid (a hallucinated arrow would be a lie); they make ornament, not evidence, and every caption that uses a generated model says so.

45.18.5 Verification and review🔗⧉

A chapter is not allowed to start drafting until it passes a readiness gate: its outline must be fleshed out, its prerequisites declared and satisfied upstream, its figures built (or stub-placeheld), its equations and notation locked, its references resolving to real documents, and its source coverage checked. Only then does prose get generated. After a draft exists, it is run past a panel of automated reviewers, each checking a different failure mode: a technical reviewer hunting for inaccuracies and missing assumptions; a pedagogical reviewer asking where a student would get lost; a prerequisite reviewer flagging background that is assumed but never introduced; a style reviewer enforcing that the whole thing reads as if written by a single author; and further checks for figure–text fit, citation correctness, and faithful coverage of the underlying slides and papers. Their findings collect into a running issues list and steer the next revision. Mechanical checks (consistent notation, language-isolated code, sane value-range and encoding assumptions) run separately and automatically. The verifiers do not replace human review; they make the human review land on the things that actually matter.

One optional step in that loop is a deliberate second opinion from a different model. Every reviewer just described is, in the end, the same family of model that wrote the prose, so it inherits the author-model's blind spots. To decorrelate the review, the pipeline can hand a slice of the book (the outline or the drafts, at chapter, part, or whole-book granularity) to an independent model from another vendor (in practice an OpenAI model) for a cross-check. Its feedback returns in a fixed two-part shape (a high-level summary, then an itemized issues list), with each issue tagged for triage: small, unambiguous corrections the editing AI applies directly, and judgment calls escalated to the human editor. The tool is deliberately read-only and sandboxed (it runs no version control and writes only its own report), so it can run alongside the main build without interfering. The value is decorrelated review from a different model, which tends to catch exactly what the resident one is systematically blind to.

45.18.6 Keeping a long book coherent🔗⧉

The hardest problem in generating a book piece by piece is that the pieces must add up to one object. Three measures hold it together. First, a set of single-source registries: one master notation table owns every symbol and its meaning, one glossary owns every key term's definition, and a registry of the book's recurring "big lessons" (that much of imaging is multiplicative; that color is non-orthogonal and non-negative; that the right basis diagonalizes a problem; that edge-preserving filtering is really about affinity; that quantization is rarely the real culprit) records where each principle first appears and everywhere it recurs. Because symbols, terms, and themes are defined in exactly one place and referenced everywhere else, they cannot drift. Second, a dependency graph: each chapter declares what it provides and what it requires, and the build checks that nothing is used before it is introduced, and propagates a change in a shared definition to every chapter that leans on it. Third, version control: every change to both the prose and the tooling is committed, so the generated and the edited versions sit side by side, edits can be extracted as differences, and the whole history is recoverable. A verbatim log of every instruction and a dated process changelog accompany the repository, so the method itself (which evolved continuously as we learned) can be reconstructed and explained, which is much of what this appendix is built from.

45.18.7 The toolchain🔗⧉

The prose is written once in a neutral markup and rendered to several outputs (print/PDF and a web edition) from a single source, so it is proofread once rather than per-format. The same single-source trick produces the book's two flavors, one with Python code and one with C++: the shared explanation is written once and language-specific snippets are swapped in per build, so a reader ever sees only one language, never a confusing both. Derived artifacts (the slide digests, the figures, and eventually the compiled chapters, the index, and the glossary) follow an incremental, build-style discipline: each records the inputs it was built from and is rebuilt only when those inputs actually change, so a small edit does not trigger a wholesale regeneration. The book even carries a version number that ticks up on each recompile.

45.18.8 What the machine did, and what it did not🔗⧉

The division of labor needs stating precisely, because "written by AI" is both an overstatement and an undersell. The AI was extremely good at the things that are laborious but well-specified: drafting fluent prose from a tight, source-backed outline; writing and debugging the figure-generation code; performing the mechanical bookkeeping (notation tables, cross-references, consistency sweeps) that no human enjoys; and building the very tooling that ran the process. It made drafting much faster. It was not trusted with judgment. A human chose the book's structure and argument, decided what mattered and what to cut, supplied the taste, caught the subtle errors that survive plausible-sounding prose, and owns every claim that remains. The safeguards described above (source-grounding against fabrication, fresh per-section context against drift, single-source registries against incoherence, and a reviewer panel feeding a human editor) exist precisely because an unsupervised model would fail in each of those ways. In plain terms, the AI drafted prose, built tooling, and handled bookkeeping, and the responsibility, like the byline, stays human. The recurring prompt patterns that made the model dependable, and the full verifier suite (V1–V24) that enforces these safeguards as runnable checks, are documented one level deeper in the companion appendix Chapter 45.22, Under the hood: prompts, patterns, and verifiers.

45.18.9 Who wrote what: a per-part estimate🔗⧉

Here is a good-faith attempt to quantify the division of labor: part by part, for the outline (the structure and content) and for the figures. "Authorship" of the outline does not cleave cleanly: the human author set the structure, chose every topic, and supplied the course material the book grows from, yet a great deal of the connective outline text (the sidebars, the cross-references, the worked framings) was drafted by the AI from those seeds and then edited. So a low "human %" on the outline does not mean the human was absent; it means the human directed and the machine expanded. Conversely, a figure counted as "AI-originated" was still requested in general terms, reviewed, and kept or killed by the human. The percentages below are estimates reconstructed from the verbatim prompt log, rounded to the nearest 5%, and meant to convey proportions, not precision.

Two columns, two questions:

• Outline (human %): of the section's outline content, how much originated as the human author's explicit direction, structure, or course material (versus AI expansion of those seeds).
• Figures (human-specified %): of the figures, how many were specifically named or described by the human (e.g. "make an interactive BRDF with a lit sphere", "use the no-parking-sign Bayer mosaic", "a wave simulation of diffraction") versus those the AI proposed on its own from a general prompt like "suggest the figures this section needs."

Two patterns stand out in the table. The human share of the outline is highest exactly where the book leans on the author's own teaching and research (Fundamentals, Basic, Optics, Multiple-exposure, Performance/Halide, and the problem-set Appendices), all rooted in the MIT course's lectures, slides, and assignments; it is lowest in the survey-style later parts, where the AI expanded thin topic lists into structured outlines that the author then pruned. The human share of the figures is concentrated in the parts that have actually been drafted and iterated (Fundamentals and Basic), where many figures were requested by name, and high again in the Appendices, whose figures are the author's own course-handout illustrations. Across the whole book, roughly a third of the outline's substance traces to explicit human direction (with the rest AI-expanded under that direction), and about a fifth of the figures were specified in detail by the author (with the rest proposed by the AI and then accepted, revised, or rejected). Every page, figure, and number, whoever first drafted it, was reviewed and is owned by the human author.

Word counts. A blunter proxy for the division of labor is simply how much text exists at each layer, kept as three separate tallies, because they are three genuinely different things and lumping them together flatters the human. (1) The pre-existing course material the book distills, all authored over years of teaching before any of this writing began: the MIT lecture transcripts (≈305,000 words; automatic captions of the recorded lectures, captured by MIT's automatic lecture-capture system and machine-transcribed), the slide decks (≈114,000 words), and the problem sets (≈42,000), about 460,000 words in all. (2) The direction typed during the writing itself (every prompt in the verbatim log), about 87,000 words. (3) The AI-generated book text produced from those: the outline (≈247,000 words) and the drafted chapters (≈969,000), about 1,215,000 words together. The drafted prose is roughly 11× the human's typed direction, and the outline-plus-prose about 14×, expanded under that direction and grounded in the much larger pre-existing corpus, not typed from a blank page. An earlier version was wrong here: it folded the outline into "human input" and reported a 2.2× ratio. That was wrong. The outline's words are overwhelmingly the model's (the human share of its substance, as the table above estimates, is about a third), so it belongs on the generated side of the ledger alongside the prose, not the human side. This is the point of the appendix: the load-bearing human contribution is the curation (the structure, the choices, the corrections, the sign-off), not the keystroke count. All four numbers are recomputed mechanically (word counts over the transcripts, slides and psets; the prompt log; and Slopendium/.md plus Manuscript/Draft — .md) so they can be maintained as the book grows.

Tokens and electricity. The word counts above tally the kept text; the generated total is far larger, because making this book meant drafting and re-drafting every chapter, running each past a panel of AI reviewers, and spawning sub-agents that reasoned at length over the figures and the tooling. The final artifacts themselves (the ≈1.2 million words of outline and prose, plus the roughly thousand figure programs and several dozen interactive demos) are on the order of a few million tokens. But counted end to end (drafting, multi-pass review, agent reasoning, and retries), the model usage is far more, and here we do not have to guess: Claude Code's own all-time usage meter puts it at about 99 million tokens, call it 100 million. (A caveat on what that counts: it is the metered token total; the gross amount the model attended to is larger still, because the book's context is re-read on many turns and served largely from a prompt cache, but the metered ≈100 M is the checkable figure, and an earlier draft's "on the order of a billion" was a loose over-estimate, off by roughly 10×.) Put very roughly into energy: at a (wide, uncertain) ~0.1–1 watt-hour per thousand tokens for large-model inference, 100 million tokens is somewhere around 10 to 100 kWh of electricity: call it a few tens of kilowatt-hours as a middle guess. To make that tangible: streaming high-definition video costs roughly 0.1 kWh per hour, so the whole book's generation is in the ballpark of a few hundred hours of Netflix, i.e. comparable to one household streaming a couple of hours every evening for a few months. Two caveats: this counts inference only, not the (enormous, amortized-over-everyone) cost of training the model; and the per-token energy of frontier models is not public, so the bracket is deliberately an order of magnitude wide. The point is not the precise number but the scale: a book that would have taken a single author many years of solitary writing was instead compiled with an energy budget that is modest relative to typical household media use.