1.3 How this book was made⧉
This book was written with AI assistance. It grew out of Frédo's computational photography course at MIT; the models helped turn that material into chapters, figures, and the software used to publish them. The author remains responsible for the book's content. The detailed accounts are Chapter 45.18, How this book was created (the early process) and Chapter 45.19, How this book is made now.
Where it came from. The book grew out of Frédo's computational photography course at MIT. Two decades of lectures, slides, and problem sets are its source material. The initial corpus contained roughly 305,000 words of lecture transcripts (automatic captions of the recorded lectures, captured by MIT's lecture-capture system and machine-transcribed), about 114,000 words of slide decks, and some 42,000 words of problem sets, or about 460,000 words of teaching, accumulated over years, before drafting began.
How it was written. The initial collaborator was Claude. The outline, prose, figure programs, and build tooling developed together. The workflow changed as draft chapters became available.
The first workflow resembled a compiler. An outline held what the book says (its structure, its figures, its equations, its sources) and a compile step turned that into prose, with the author's edits triaged back upstream so that recompiling would retain the corrections. That worked while the structure was being developed. It stopped being how anyone actually worked once substantial drafts existed, because editing an existing page proved more convenient than regenerating it. So the drafts became the source. The outline is still there, and it still carries structure and sources, but for a chapter that has been written the prose is now the text being edited, in place, by describing the change rather than by regenerating the chapter.
Specialized agents now handle prose, figures, software, and other parts of the project. They coordinate through repository files: an ownership map, a shared process, per-agent state files, and inboxes for handoffs. This arrangement helps prevent two agents from changing the same material without knowing about each other. Many figures are generated from programs, which makes them reproducible, but code and captions can still disagree. Both need review.
How it is organized. Applications motivate the tools, but a reader needs some tools before an application can be explained. Denoising, for example, draws on probability and filtering; those subjects are easier to understand with a noisy photograph in view. The book alternates between methods and applications and uses cross-references where a single reading order cannot supply every prerequisite. Historical order offers another perspective, but later concepts often provide a clearer explanation of earlier inventions.
Who did what. Models drafted from sources, wrote figure code, and maintained notation and references. The author chose the structure, decided what to emphasize, and reviewed the explanations and results. Fabrication remained a risk; concurrent agents added coordination failures, including overlapping edits, incomplete handoffs, and checks run against changing files. The process records ownership and work state to make these failures easier to detect. Passing a mechanical check does not establish that a claim or figure is correct.
A note on the numbers. An accounting snapshot recorded about 92,000 words of typed prompts and 1,417,000 words of generated outline and prose, roughly fifteen generated words per prompt word. These counts describe text volume; they do not measure authorship or quality. The outline was model output too; an earlier calculation incorrectly counted it as human writing. Session and token totals depend on how repeated context is counted, so the detailed accounting belongs in Chapter 45.19, How this book is made now. The human contribution includes the teaching material, structure, selection, corrections, and review, much of which cannot be measured by counting new keystrokes.
Did writing it this way make me dumber? I worried that relying on a model might reduce my expertise. My experience was the opposite. Building the book pushed me to examine details I had passed over in years of teaching: how a Poisson solve changes in linear or gamma-encoded space, where half-pixel resampling conventions matter, and which familiar claims actually survive a small experiment. The models made these experiments easier to set up.
The collaboration also helped me reconsider the structure of the field: which ideas depend on each other, where physical models enter learning, and which concepts deserve more attention. Deciding what belonged in the book led me to conversations with researchers and engineers across academia and industry. Those conversations changed the table of contents and taught me things I would not have sought out on my own.
The benefit depended on staying involved in the reasoning. A plausible answer still needed to be checked and understood. The text counts describe how much material the process generated; they do not measure what I learned from reviewing it.
Several details of that experience matter to me. Running small experiments became cheap enough that I could examine questions I had previously waved past: Poisson reconstruction, gradient blending, and tone curves in linear, gamma, and log coordinates, or the effect of half-pixel sampling conventions. The experiment often taught more than another plausible paragraph would have.
Organizing the material required a different kind of work. A chronology would show which inventions made others possible, but would scatter related ideas across decades. A conceptual organization brings those ideas together, sometimes using a later interpretation to explain an earlier invention. There is no single sequence that does both jobs well. The cross-references and repeated applications are part of how we handle that compromise.
The conversations with researchers and engineers were another consequence of the project. Asking what belonged in the book led to asking which ideas practitioners should understand but often do not. Those conversations changed both the coverage and my understanding of the field. The quantity of generated text does not measure that benefit. The value of the assistance was that it left more time for these experiments, conversations, and editorial decisions.
Coordination created its own costly failures: an agent could finish work without handing it over, modify a file another agent owned, or run a check while its input was changing. The most expensive way to discover any of these was for me to notice the problem in the book. State files, ownership boundaries, and explicit handoffs exist because a plausible report of completion is not enough.