3.5 The visual system attends little and makes things up🔗⧉

While looking for a person in a photograph, you can miss a conspicuous object elsewhere. Vision selects information for a task, with much finer spatial detail near the point of gaze. Attention and inference help explain that selectivity, but the peripheral input is not empty.

3.5.1 Attention and eye movements🔗⧉

The eyes alternate fixations with rapid movements called saccades, and use smooth pursuit to follow moving objects. A scanpath records these movements over time (Figure 3.5.1⛶ fullscreen). Faces, text, contrast, and the observer's task can influence fixation locations. Attention can also shift without moving the eyes, so gaze is an incomplete measure of what a viewer processes.

The examples show selected observers and images. Different tasks can produce different scanpaths on the same photograph.

What draws the eye (saliency) can be modeled, and modern saliency and gaze-prediction models are learned with machine learning (→ the saliency / gaze-prediction material in the Part 7, Probabilistic approaches and machine learning). Saliency in turn feeds content-aware image operations: where can we crop or retarget an image without disturbing what the viewer will actually look at? That is the link to auto-cropping and seam carving later in the book.

Long before any of this was measured, artists were doing it by hand. Guiding the viewer's attention (controlling where the eye goes, and in what order) is one of the central things a painter, photographer, or cinematographer does, and it is precisely what composition is for: leading lines, contrast and brightness, selective focus, color accents, framing, and the placement of the subject all steer the gaze toward what matters and away from what doesn't. The scanpath and saliency map are simply the empirical, measurable face of a craft that has always been about directing attention (→ the composition tactics in Part 40, Human factors).

3.5.2 The visual system makes things up🔗⧉

Peripheral vision resolves less fine detail and is especially affected by crowding, interference from nearby features Bouma 1970 (crowding). It still supplies useful information about layout, objects, and motion Rosenholtz 2016 (peripheral vision). Sensitivity also changes around saccades, but saccadic suppression is not complete blanking. These constraints support inference and selective sampling rather than a literal, uniformly detailed internal photograph. Hertzmann discusses the resulting impression of broad awareness Hertzmann 2024 (blog: illusion of awareness).

In inattentional blindness, an observer occupied with one task may fail to report an unexpected event, as in the basketball-pass experiment of Simons and Chabris Simons & Chabris 1999 (invisible gorilla). In change blindness, a substantial change can go unnoticed across views Rensink, O'Regan & Clark 1997 (change blindness). Simons and Levin even changed a conversation partner while a door briefly occluded the interaction Simons & Levin 1998 (the door study). Such results demonstrate limits on noticing and comparing information; they do not show that everything outside a fixation is invented. Everyday failures of noticing are discussed by Wolfe and colleagues Wolfe et al. 2022 (normal blindness).

Foveated rendering can reduce detail away from measured gaze, but it must allow for eye-tracking error and latency. Saliency-based cropping can preserve likely subjects, but the intended task and composition still matter. Compression and retargeting need evaluation on their actual artifacts; inattentional blindness is not a general license to discard whatever a model labels unimportant.

The basketball demonstration is useful because the visible event is not small: while observers count passes, a person in a gorilla suit crosses the scene. The task directs attention toward one kind of event, and in the original study about half the observers failed to report the unexpected one, with the rate depending on the task's difficulty Simons & Chabris 1999 (invisible gorilla). In the door experiment, a passing door briefly interrupts a conversation and conceals an exchange of the person giving directions Simons & Levin 1998 (the door study). Here the issue is comparing identity across an interruption, and about half the pedestrians did not notice that they were now talking to someone else. These demonstrations probe different failures of noticing; neither requires the periphery to contain no useful information. In a photograph, a viewer searching for a face and a viewer inspecting compression artifacts are likewise performing different tasks, so a saliency map alone cannot specify which errors will be noticed.

This matters to how we compare a photograph with seeing. The photograph is not a degraded copy of a complete internal picture: vision samples, attends, and interprets. A viewer can miss a conspicuous change while following a subject, then notice it immediately when asked to inspect that region. An imaging method should therefore be evaluated under the viewing task it is intended to support. The same selectivity that permits perceptual compression also makes a casual glance an unreliable test of reconstruction accuracy.

Recap: big lessons of this chapter🔗

💡 Big lesson (L3.19) — vision is foveal and serial: the eye scans🔗

Eye movements bring selected regions onto the high-acuity fovea. Peripheral signals also support scene understanding and guide the next fixation. Gaze depends on the task as well as image features; a fixation map is not a complete map of attention.

💡 Big lesson (L3.20) — seeing is a construction, not a recording🔗

Perception combines incomplete measurements with context, attention, and inference. Change blindness, inattentional blindness, and crowding expose limits on what we notice and retain. They do not justify treating unfixated image regions as unobserved or irrelevant.