42.0 SYSTEMS⧉
An algorithm in this book becomes a photograph only after it runs inside a system: a camera that lets software choose each exposure, a library that supplies the primitives, an editor that stores what the user did, a network that moves the result, and a phone API that decides what an application may touch. Each of these makes design decisions the algorithm never sees and still depends on. A bilateral filter written against whole images in memory does not run on a 100-gigapixel canvas; a burst merge cannot run on a camera that returns only its finished JPEG.
This part reads those systems from the inside, preferring ones whose source we can inspect. Chapter 42.1, Programmable and modular cameras starts at capture, with the per-frame request model that made burst photography possible and the attempts to make camera hardware modular. Chapter 42.2, Image processing libraries surveys what the primitive libraries provide and the design axes that separate them. Chapter 42.3, Lightroom-style raw developers and Chapter 42.4, Photoshop-style editors take apart the two dominant editing models, each through an editor written alongside this book: CompPhotoRoom, the browser raw developer of Chapter 8.13, A raw developer in the browser, and gamut, a layer-based native editor. Chapter 42.5, Networking and image transport covers delivery formats and the question of where to compute. Chapter 42.6, Photography programming on phones closes with the Android and iOS interfaces through which most computational photography reaches people.
The two editors were designed separately and converged on the same architecture. Both keep one processing graph and drive it three ways: a small proxy for every slider drag, a tile around the visible region at full resolution, and a streamed pass for export. Both put module boundaries at algorithms rather than at operators, because the cost of a boundary is the intermediate image it writes to memory. Both treat a parameter as a runtime value, so dragging a slider never recompiles anything. These choices recur because the constraint behind them recurs: moving pixels through memory costs more than computing on them, a theme Part 29, Making computational photography code fast develops in depth.
Contents of this part
- 42.1 Programmable and modular cameras
- 42.2 Image processing libraries
- 42.2.1 What a library provides
- 42.2.2 Eager whole-image libraries
- 42.2.3 Demand-driven, streaming libraries
- 42.2.4 Separating the algorithm from its schedule
- 42.2.5 GPU and tensor libraries
- 42.2.6 Choosing
- 42.3 Lightroom-style raw developers
- 42.4 Photoshop-style editors
- 42.4.1 The layer stack
- 42.4.2 Tiles
- 42.4.3 Case study: gamut
- 42.4.4 Scheduling: pull, cache, fuse
- 42.4.5 What it buys
- 42.4.6 Editors in the browser
- 42.5 Networking and image transport
- 42.6 Photography programming on phones