BookNotesSlidesP-sets Syllabusdraft · v0.2.49.1
💬Comments welcome. To leave a note, select any text and click the note / highlight button that pops up — or open the panel with the tab at the top-right (‹). Notes are visible only inside our private review group.
Computational Photography, an AI-powered Slopendium — 03 Visual perception and color
expand to📖 Full book outline1 parts · 7 chapters · 40 sections · 33 figures embedded · 1 placeholders · double-click a figure to enlarge
▸Part 3 VISUAL PERCEPTION AND COLOR⧉
fig-light-journey
fig-light-journey · FUNDAMENTALS part opener — the journey of light: source → scene (reflection) → lens → sensor / retina → visual system, each step labeled with its chapter ✅
<!--uid: visual-perception-and-color-->
This part picks up where [[Fundamentals of imaging]] leaves off: after light has been focused by a lens onto the retina or a sensor. Where part 2 treated the image as a physical measurement, this part asks what that measurement *means* to an observer. Color is not a property of light but a projection made in the eye and brain, and that single fact grounds white balance, color management, JPEG's chroma subsampling, and tone mapping downstream.
• **The eye and image formation** — the eye as an optical instrument: cornea, pupil, lens, and retina; the cone/rod mosaic and the fovea; and the eye's own refractive errors and how they are measured (inverse imaging).
• **Perceptual color and trichromatic vision** — why three cones make us trichromatic; color as the projection of a spectrum onto three numbers; metamers, color blindness, opponent processing, and color as linear algebra.
• **Adaptation and constancy** — what the visual system does with light once it has it: light adaptation, working in ratios (contrast), and discounting the illuminant to report surface lightness and color (constancy).
• **Spatial and temporal vision** — how vision resolves detail across space and time: the contrast sensitivity functions, acuity, and flicker fusion, the limits every codec and display is tuned against.
• **The visual system attends little and makes things up** — attention and eye movements, and the constructed, filled-in nature of what we see.
• **Animal eyes** — human vision is one solution among many: how other eyes form images and sense color, from compound eyes to tetrachromats to the mantis shrimp.
▸3.1 The eye and image formation⧉
fig-eye-cross-section
fig-eye-cross-section · eye cross-section 🟨
fig-retina-layers
fig-retina-layers · retina layers & photoreceptor synapses 🟨
fig-cone-rod-distribution
fig-cone-rod-distribution · cone/rod distribution & fovea 🟨
fig-visual-pathway
fig-visual-pathway · pathway retina→LGN→V1 🟨
fig-photoreceptor-cell
fig-photoreceptor-cell · anatomy of a rod & cone cell 🟨
fig-rods-cones-micrograph
fig-rods-cones-micrograph · micrograph of rods & cones (primate retina) 🟨
• light, lens, pupil, retina,
• cones/rods, distribution, fovea — note there are **no S (blue) cones in the very center of the fovea**, and far more L+M than S overall, so **blue is low-resolution and a bit out of focus** (chromatic aberration) — yet we perceive blue everywhere (the brain fills it in). Luminance ≈ the **L+M cone sum**, which is why **green carries most of the acuity** (and why Bayer uses 2× green).
• cells, ganglions, on-retina spatial processing, visual cortex, V1, higher.
• **the eye as a camera — and its refractive errors.** Like any lens system the eye can mis-focus: **myopia** (eyeball too long / cornea too curved → image forms *in front* of the retina), **hyperopia** (too short → *behind*), **astigmatism** (a non-spherical cornea → two focal lines, not one point), and **presbyopia** (the aging lens stiffens and loses **accommodation**). Each is corrected by a lens quoted in **diopters** (D = 1/focal-length in meters): a negative/diverging lens for myopia, positive/converging for hyperopia, a **cylindrical** lens (with an axis) for astigmatism. [`fig-refractive-errors`]
• **how the prescription is *measured* (the machines), and why it's just inverse imaging.** **(1) Retinoscopy** — shine a moving streak of light into the eye and watch the **retinal reflex** sweep across the pupil; its direction and speed reveal myopia vs hyperopia, and trial lenses are stacked until the reflex stops moving ("neutralization"). **(2) Autorefractor** — automates this objectively: it projects an **infrared** target onto the retina and analyses the **returning image** (its focus/size, or a Hartmann–Shack / Scheimpflug pattern) to solve for the corrective lens that makes the retinal image sharp — essentially a little camera running autofocus *on your eye*, in seconds. **(3) Phoropter** — the familiar "better one… or two?" wheel of trial lenses over an eye chart: a **subjective** refinement of the machine's objective estimate. **(4) Wavefront aberrometer (Shack–Hartmann)** — for the full optical picture: a **lenslet array** samples the wavefront leaving the eye and measures **higher-order aberrations** (coma, spherical, trefoil), not just sphere/cylinder — the same wavefront sensing used in **adaptive optics** and in planning **LASIK**. The throughline: measuring an eye is **inverse imaging** — find the optics that would make the retinal image (or the returning wavefront) ideal. (Cross-ref wavefront / Shack–Hartmann sensing and adaptive optics in [[Slopendium/Compound lenses, and aberration correction]].)
▸3.2 Perceptual color and trichromatic vision⧉
3.2.1 Cone photoreceptors and perceptual color⧉
3.2.2 Metamers⧉
3.2.5 Color as linear algebra⧉
▸3.3 Adaptation and constancy⧉
• the perceptual machinery the visual system runs *on top of* color: adapting to level, working in ratios (contrast), and discounting the illuminant to report surface reflectance (lightness and color constancy), the properties compression, tone mapping, and display design optimize for.
3.3.2 Visual perception is largely invariant to multiplications⧉
3.3.5 Discounting the illuminant⧉
3.3.6 Failure of color constancy when viewing photographs⧉
▸3.4 Spatial and temporal vision⧉
3.4.2 Chrominance spatial vision⧉
▸3.5 The visual system attends little and makes things up⧉
3.5.2 The visual system makes things up⧉
▸3.6 Animal eyes⧉
fig-eye-evolution
fig-eye-evolution · the evolution of the eye, in cross-sections: flat photoreceptor patch (no image) → cup (directional) → pinhole (an image, no lens) → lensed eye — recapitulating sensor → camera-obscura → lens (Bonus: animal eyes) 🟨
• **the eye evolved as a sequence of small improvements**, and it recapitulates the optics of this book — from "no image" to a focused one. Following Nilsson's stages (Land & Nilsson, *Animal Eyes*):
• **a flat patch of photoreceptors** (an *eyespot*): just **photosites**, **no image formation** — it tells light from dark and, crudely, which side has *more* light (phototaxis), but cannot form a picture. This is the bare **sensor** with no optics.
• **a concavity / cup**: fold the patch into a **pit** and the rim's shading gives **directional** sensitivity — the deeper the cup, the better you can tell *where* light comes from.
• **a pinhole**: close the cup almost shut → a **pinhole eye**, a real (if dim) **image** with no lens at all (the **nautilus** still uses one) — exactly the camera obscura of [[#Pinhole image formation and linear perspective]].
• **a lens**: fill the aperture with a **refractive lens** (often a graded-index sphere) → a **bright, focused** image, independently evolved in fish, cephalopods, and vertebrates. Add a **cornea, iris, and accommodation** and you have the human eye.
• so the human eye is *one* solution; evolution reached the same goal by very different routes (below).
• **a tour of those other solutions** — a useful contrast to the human eye and to camera design:
• **compound eyes** (insects): many **ommatidia** → wide field of view and fast temporal response, but low spatial resolution
• **more cone types**: birds / reptiles are **tetrachromatic** (a 4th cone, into the UV); the **mantis shrimp** has ~12–16 photoreceptor classes (yet poor color *discrimination* — it seems to recognize rather than compare) — the **divergence of opsins** across animal groups is the color sidebar in [[#Color]]
• **tapetum lucidum** (cats, deer): a reflective layer behind the retina that boosts night vision (and causes eye-shine)
• **polarization vision** (cephalopods, many insects): they see light's **polarization** — a channel we're blind to
• **other eye designs**: **pinhole** (nautilus), **mirror eyes** (scallops — image formed by a concave mirror, not a lens), and the front-facing (**stereo**) vs side-facing (**field-of-view**) placement trade-off
• further reading: Land & Nilsson, *Animal Eyes*; the "Evolution of Eyes" chapter in *Vision* (Cambridge); a friendly overview at phos.co.uk, "The Evolution of Sight".
3.6.1 The optics: many ways to form an image⧉
3.6.2 The pupil: aperture shape as a clue to ecology⧉
3.6.3 Amphibious vision: focusing in air and in water⧉
3.6.4 The sensor: a retina that is not uniform⧉
3.6.5 Seeing in the dark: sensitivity and its tricks⧉
3.6.6 Acuity: the finest detail, and how to sharpen it⧉
3.6.7 Color vision: opsins remixed⧉
3.6.8 Polarization: the axis we are blind to⧉
3.6.9 Temporal vision: flicker, and motion first⧉
▸3.7 Human vision TLDR⧉
3.7.1 The eye is a camera with an uneven sensor⧉
3.7.2 Three cones reduce a spectrum to three numbers⧉
3.7.3 Primaries are a choice, and the axes are skewed⧉
3.7.4 Color blindness is a missing or shifted axis⧉
3.7.5 After the cones, brightness and two color differences⧉
3.7.6 The eye works on ratios, not levels⧉
3.7.7 The eye reports surfaces, not light⧉
3.7.8 Spatial vision is band-pass, and measured in degrees⧉
3.7.9 Color detail is coarser than brightness detail⧉
3.7.10 Vision has a finite temporal bandwidth⧉
3.7.11 Vision is foveal, serial, and partly invented⧉
3.7.12 What the rest of the book does with this⧉