BookNotesSlidesP-sets Syllabusdraft · v0.2.49.1
◀▲⌂▶
💬Comments welcome. To leave a note, select any text and click the note / highlight button that pops up — or open the panel with the tab at the top-right (‹). Notes are visible only inside our private review group.

19.3 Blue and green screen matting, chroma key⧉

A controlled screen reduces uncertainty about the background and separates its color from much of the subject. That makes matting easier, but knowing the background alone still leaves four unknowns, foreground RGB and alpha, in three observed channels.

19.3.1 Blue/green-screen matting and chroma keying⧉

In $C=\alpha F+(1-\alpha)B$, fixing $B$ leaves a family of possible $F,\alpha$ pairs. Chroma keyers add assumptions about foreground colors and screen contamination. Distance from the key color is a useful heuristic, not a general measurement of fractional coverage. Smith and Blinn analyze these constraints and the remaining ambiguity (Smith & Blinn 1996).

Much of the practical method comes from one engineer. Petro Vlahos developed the color-difference blue-screen process for film in the late 1950s, which estimates coverage from how much the blue channel exceeds the others rather than from a threshold on blue alone, and so keeps semi-transparent edges such as hair and motion blur. It was used on Ben-Hur (1959). He later worked on Disney's sodium-vapor process (Chapter 35.3, Illumination-based matting) and in 1976 co-founded Ultimatte, whose electronic keyers brought color-difference keying to video. The formulas Smith and Blinn analyze are generalizations of his.

A second known backing can supply the missing constraint if foreground appearance and coverage remain unchanged. Subtract two captures:

$$C_1-C_2=(1-\alpha)(B_1-B_2).$$

Writing $d=B_1-B_2$, a least-squares estimate is

$$1-\hat\alpha=\frac{d^{\mathsf T}(C_1-C_2)}{d^{\mathsf T}d}.$$

The backgrounds must differ enough for a stable denominator. Motion, spill changes and illumination changes violate the shared-foreground assumption. A bounded estimate can be projected to $[0,1]$, with residuals retained as a consistency diagnostic. For positive alpha, foreground recovery divides by alpha and becomes unstable near zero.

Choose a screen color separated from the subject and compatible with the camera's channel sensitivity, noise and sampling. Bayer arrays sample green more densely, but that alone does not prove it is always the quietest or best key channel. Wardrobe, spill and the capture pipeline can favor blue or another backing.

Spill changes the foreground itself through colored illumination. De-spill is an appearance correction, not guaranteed recovery by subtracting one fixed green value. Uneven backing can be modeled with a clean plate when available; shadows, wrinkles, reflections and motion complicate it. Keep the subject away from the screen when practical, expose channels without clipping, and inspect hair against light and dark replacement backgrounds.

fig-chroma-key
Figure 19.3.1. Chroma-key matting, interactive (camera-capable). Pull an alpha matte from how close each pixel is to the key color and composite the subject over a new background. Choose green or blue (or click to eyedrop a custom key), then set the tolerance (where $\alpha$ crosses ½) and softness (the width of the soft edge), and watch the matte and the composite update. Spill suppression removes the screen light that bounced onto the subject's edges; a matte view shows $\alpha$ directly. The new background can be a blue screen, a white field, a black field, a checkerboard (to expose fringing), or a photo backdrop, and the panels run input → matte → final composite. Run it on the built-in green-screen frame, upload an image, or (best) point your camera at a plain wall and eyedrop the wall color to key yourself out live. This is the engineered easy case of the matting equation, $B$ known by construction; the hard, ordinary-background case is the rest of the chapter.

Controlling the backing solves only part of a hair pixel. The screen color may be known, but the pixel still contains a mixture with the hair's unknown color and fractional coverage. Thresholding it into either foreground or background discards that mixture and can produce a hard or broken outline. Screen spill adds another problem: it changes the light reflected by the hair, so removing the background mixture alone can leave a green foreground edge. These two errors call for different operations, estimating a matte and correcting contaminated foreground appearance. Inspecting the result on both bright and dark replacement backgrounds helps reveal what one backdrop hides.

The broader strategy is to change the capture so that fewer unknowns remain. A controlled backing supplies information about background color. Chapter 35.3, Illumination-based matting explores other measurements that separate foreground from background. When neither capture control nor additional measurements are available, natural-image matting must depend more strongly on user constraints and priors.