3.0 BASIC IMAGE PROCESSING AND ISP⧉

The first part of this book put a camera in front of the world and watched light turn into a grid of numbers. From here on, that grid is the world we work in. We have a photograph in memory, and the question is no longer where the photons came from but what we can do with the numbers they left behind. By the end of this part you should be able to build the core of a small photo editor — a mini Lightroom with exposure, contrast, color, sharpening — or the camera-side version of the same machinery, the image signal processor that turns raw sensor data into a finished JPEG.

A great deal lives in here, it turns out. Almost every operation you associate with a photo app or a camera is in this part: making a picture brighter or punchier, fixing its color, sharpening it, blurring it, scaling it up or down, cleaning up noise, saving it small enough to send. Underneath that variety sit a handful of ideas that recur the whole way through — and we will meet them in an order designed so each one earns the next. We start with the representation itself: what the array is, what its numbers mean, and the small conventions (coordinates, channels, edges) that quietly cause most beginner bugs (this chapter). Because image code is famously easy to get subtly wrong, we then spend a short chapter on how to develop, test, and debug it — the habits that save you later. With that in hand we look at point operations (a curve applied to every pixel: exposure, contrast, tone, histograms), then tone mapping for scenes whose range outruns the display, then neighborhood operations and convolution (a pixel computed from its neighbors: blur, sharpen, gradients). Convolution opens the door to Fourier, which gives us the language for sampling, aliasing, and resampling, and the pyramids that represent an image at many scales at once. We close with the pieces a real camera bolts together: file formats and compression (how a JPEG actually works), the metrics by which images are compared, denoising and demosaicking, and finally the image signal processor — the fixed pipeline from raw sensor data to a finished JPEG that ties every chapter of this part into one block diagram.

One thread runs through all of it, and it is worth stating up front because it will save you grief: the same array of numbers can mean different things, depending on how it is encoded. Do the physics — combining exposures, white balance, blurring — on linear values that are proportional to light; do the display and the compression on gamma-encoded values tuned to perception; and reach for log when the range is enormous. Knowing which of these a given operation wants is half of getting image processing right. This chapter raises that flag; the next chapters pin it down.

Contents of this part

▸ full collapsible outline of this part

3.0 BASIC IMAGE PROCESSING AND ISP🔗⧉

Contents of this part

3.0 BASIC IMAGE PROCESSING AND ISP⧉