BookNotesSlidesP-sets Syllabusdraft · v0.2.49.1
◀▲⌂▶
💬Comments welcome. To leave a note, select any text and click the note / highlight button that pops up — or open the panel with the tab at the top-right (‹). Notes are visible only inside our private review group.

5.1 The camera as an instrument⧉

A child runs into a dim room. The camera can choose a bright rendering, yet return a blurred face; a phone can deliver a clean room with an oddly reconstructed hand. Before choosing settings, decide what the capture must preserve. In this case it is the person's expression and motion, not every detail in the furniture.

The useful questions about a camera are operational: which settings can we control, what feedback does it provide, how quickly can it make the next exposure, and what processing happens before we receive the file?

5.1.1 UI and display⧉

An optical viewfinder shows the scene through an optical path. It can be easy to follow action, but it does not preview the recorded brightness or white balance. An electronic viewfinder or rear screen shows a processed live image and can overlay a histogram, highlight warnings, focus peaking and a magnified crop. Its brightness may be boosted in dim light or fixed for flash work. Treat it as feedback from a particular preview pipeline.

The optical finders are worth understanding because each solves a geometric problem. A single-lens reflex puts a mirror at 45 degrees behind the lens that folds the image up onto a focusing screen at the same optical distance as the sensor, so what is sharp on the screen is sharp on the sensor, with no parallax. That screen image is upside down and, after the mirror, reversed left to right. The pentaprism above it, a five-sided glass prism with a roof of two faces at 90 degrees, reflects the light so that it leaves the eyepiece upright and the right way round. Cheaper bodies use a pentamirror, the same geometry built from a hollow shell of mirrors: lighter and less expensive, but dimmer, since each metal-mirror reflection loses more light than a reflection inside glass. At the moment of exposure the mirror flips up and the finder goes dark. Cameras without a reflex mirror used a separate direct finder, a small reverse-Galilean telescope beside the lens, which frames the scene but sees it from a slightly different position, so its framing drifts from the lens's up close and it shows neither focus nor exposure; the rangefinder adds bright-line frames and a focusing patch to that window (Chapter 22.8, Focus). The waist-level finder of a twin-lens reflex or a medium-format camera is the most direct of all: you look down onto the focusing screen with no erecting prism, so the image is upright but reversed left to right, and panning to follow a subject feels backward until you learn it.

Check focus on the intended subject, then inspect the brightest area whose detail matters. A histogram of the preview can clip before a raw channel does, or hide a channel problem behind tone mapping. Zebras mark a configured signal level, which need not be raw saturation. Learn that relationship with a bracketed test of your own camera. Review a captured frame as well as the live feed: the final aperture, exposure duration and multi-frame processing may differ.

fig-viewfinder-types
Figure 5.1.1. OVF vs EVF vs rear LCD. The OVF (SLR mirror/prism or rangefinder window) shows the real world directly, with zero lag, but not the captured exposure or white balance. The EVF and rear LCD show the sensor's live feed, slightly laggy, but a close preview of exposure, white balance, and depth of field (still a processed preview, not the final file), plus overlays (live histogram, focus peaking, zebras).

5.1.2 Anatomy of a modern full-frame interchangeable-lens camera⧉

Light passes through the lens and, when present, the mechanical shutter before reaching the sensor. The sensor converts it to charge and reads it out; the processor estimates focus and exposure, develops a preview and writes image data to storage. A mechanical shutter controls the light physically. An electronic shutter controls integration and readout electronically. Neither the word “mirrorless” nor a silent mode guarantees simultaneous exposure across the frame.

An interchangeable mount provides access to different fields of view, working distances and apertures. Some bodies move the sensor for stabilization; some lenses move an optical group. Controls and a grip make it possible to change settings while following a subject. Buffer capacity, card throughput and battery endurance determine how long that workflow can continue. A high advertised burst rate is of little help if focus locks on the first frame or the buffer fills before the event.

fig-mirrorless-anatomy
Figure 5.1.2. Cutaway of a modern mirrorless body, light to bits. Lens + mount (electronic contacts) → sensor on the IBIS stage → mechanical and/or electronic shutter → image processor + AF (on-sensor PDAF, subject detect, ISP) → EVF + rear LCD → card + battery. Inset: the older SLR's mirror box and optical prism, the parts mirrorless removes. That enables WYSIWYG preview, full-frame on-sensor PDAF, and shorter flange distances.

The change from an SLR to a mirrorless camera is easiest to understand by following the viewing path. In an SLR, a mirror directs light upward toward a focusing screen and prism for optical viewing. The mirror moves out of the taking path for the exposure. A mirrorless camera instead reads the sensor to produce the electronic view. Removing the mirror box also gives the optical designer room to use a shorter flange distance, the distance from the lens mount to the sensor. This changes lens-design and adaptation possibilities; it does not require the mount itself to be smaller.

The electronic view can show a processed approximation of exposure and color before capture, while an optical view can be useful for watching the scene directly. Focus peaking marks strong image edges and magnification helps inspect the intended plane of focus. Neither replaces checking the final image: a strong edge can belong to the wrong subject, and preview settings can differ from capture settings.

5.1.3 Anatomy of cell-phone cameras⧉

A phone packages small camera modules with a processor and a display. Many modules have fixed apertures, so exposure automation mainly chooses integration time and gain. Modules with different focal lengths cover different fields of view. A zoom gesture can switch modules, crop one image or combine measurements from several; a displayed zoom ratio does not specify which operation occurred.

Burst capture can reduce noise or extend recorded dynamic range. It also introduces a time span over which the subject must be tracked. Portrait processing can synthesize defocus from estimated depth, and errors often appear around hair, glass and occlusions. These are consequences of the capture and reconstruction model, not exceptions that disappear because the interface presents one shutter button. Part 13, Denoising, Chapter 17.3, Merging exposures and Chapter 23.2, Fake (synthetic) depth of field develop the algorithms.

A folded telephoto module turns the optical path sideways with a prism or mirror so more of its length lies along the phone body. The fold provides packaging space; the lens still has to form the image and correct aberrations. Between modules, a zoom control may change from optical capture to cropping or fusion. The transition matters when comparing detail at a particular zoom setting, because a crop retains fewer measured samples of the scene.

Small modules also tend to give substantial depth of field at ordinary viewing distances. Portrait mode can then use a depth estimate to apply blur after capture, giving a kind of control the optical aperture alone does not supply. A close subject can still produce real optical background blur. Sensor size by itself is not a rule that every phone photograph is sharp everywhere.

5.1.4 Camera versus phone⧉

Compare cameras from the same viewpoint, with the same framing and exposure time, and judge at the same output size. Otherwise a change in subject size or processing can be mistaken for a sensor advantage. For equal scene radiance, f-number and transmission, a larger active sensor area collects proportionally more light across the image. In an idealized comparison,

$$\frac{n_A}{n_B}\approx\frac{A_A}{A_B}\frac{t_A}{t_B}\left(\frac{N_B}{N_A}\right)^2\frac{\tau_A}{\tau_B}.$$

Here $n$ is total collected photons, $A$ active sensor area, $t$ integration time, $N$ f-number and $\tau$ transmission. Quantum efficiency is an additional factor when comparing electrons rather than photons. The expression assumes matched field of view and scene content, with similar field falloff. It does not predict final image quality by itself.

Consider hypothetical sensors of 864 and 54 mm². At matched framing, time, f-number and transmission, their area ratio is 16. If shot noise dominates, that is a factor of four in signal-to-noise ratio at a common output resolution. But the larger format then has shallower depth of field. Closing its aperture by a factor of four in f-number to approximately match depth of field removes the 16-fold light advantage at the same time. State which comparison matters for the photograph. For scale: a phone main camera at f/1.8 with a focal length near 6 mm has an entrance pupil about 3.3 mm across, while a 50 mm f/1.8 lens has one about 28 mm across, some seventy times the area. The f-numbers match; the light gathered per second, spread over the much larger sensor, does not.

A burst can spend more total capture time, and a good reconstruction can use those measurements better than a poor one. Motion and read noise limit the benefit. Compare single frames first if the question concerns hardware, then compare complete capture modes if the question is which device gets the desired photograph. Megapixels alone settle neither question.

Interchangeable lenses offer physical choices a fixed module cannot. Phones offer convenient capture, processing and distribution in one device. Actual aperture controls, raw access, connectivity and update policies vary by model. Glass versus plastic and dedicated camera versus phone are inadequate proxies for measured performance.

5.1.5 Types of cameras⧉

Choose a system around the photograph and the conditions of use. A fixed-lens compact can be easy to carry; an interchangeable-lens body can accept a macro lens or long telephoto; an action camera can survive a location where a larger body would be impractical. Sensor format, available lenses, control access and handling are separate properties.

A drone, pole or aerial platform changes where the camera can be placed. It does not remove exposure, vibration or viewpoint constraints. Historical camera designs remain useful examples of different compromises: a view camera permits lens and film movements; a rangefinder offers an optical viewing path separate from the taking lens; an instant camera produces a physical print. The technology taxonomy is useful when it explains a capability, not as a ranking of photographers.

fig-big-camera-vs-phone
Figure 5.1.3. A full-frame interchangeable-lens camera beside a phone, photographed side by side at the same scale. Most of the size difference is sensor area and the lens that has to cover it; as the section explains, size alone does not settle which device makes the better photograph. (Author's photograph.)
fig-camera-market
Figure 5.1.4. Two disruptions in the camera market. Left: dedicated-camera shipments, film cameras collapsing as digital arrives (~2000–2005); the compact digital camera booms to ~120 million units a year and then craters once phones are good enough (~2010 on); interchangeable-lens (DSLR + mirrorless) cameras peak and settle into an enthusiast niche. Right: the same series with smartphones added on a log scale, over a billion camera-phones a year, dwarfing every dedicated category combined. Approximate worldwide unit shipments (CIPA statistics plus smartphone estimates, rounded for shape). The dedicated camera did not so much vanish as fold into the device everyone already carries. Unit shipments count devices sold, not photographs taken. That is the historical reason this is a book about computation, not optics.
fig-camera-types
Figure 5.1.5. The space of cameras on two axes. The ladder on the left is drawn to scale from real sensor dimensions, and that is the part worth looking at: a typical phone main-camera sensor is not a bit smaller than full frame, it is a twentieth of the area (phones vary), with the crop factor and area ratio beside each rung computed from those numbers. The plane on the right places the familiar categories on the two choices that actually decide things: how big the sensor is, and whether the lens comes off. Sensor size sets depth of field, light per exposure and bulk; the lens axis sets versatility and price. Everything else, the mirror or its absence, the viewfinder, the badge, follows from those two or from history.

Several independent classifications help describe the available choices.

PropertyExamplesWhy it matters
Recording formatSmall phone sensor, one-inch-type, Micro Four Thirds, APS-C, full-frame, medium or large formatChanges the sensor area and lens choices; compare light collection at stated framing and settings
Lens systemFixed lens, fixed zoom, interchangeable lensesDetermines which fields of view and working distances are available
Viewing pathReflex optical finder, separate rangefinder window, electronic viewChanges what can be previewed and how focus is judged
Camera movementsView or technical camera with shift and tiltChanges framing or the plane of focus without simply pointing the whole camera elsewhere
Recording mediumElectronic sensor, film, instant materialChanges the development and delivery workflow
Carrying platformHandheld, tripod, pole, kite, balloon, droneDetermines accessible viewpoints and the motion that must be controlled

An action camera and a studio camera can therefore differ on several axes at once. Calling one “better” leaves the intended location, output, and handling unspecified. The platform distinction is especially useful: lifting a camera changes its viewpoint without changing the principles of exposure or image formation.

A camera is also classified by how it is carried. The drone is today's default for getting above a scene, but it caps a long lineage: Nadar photographed Paris from a tethered balloon in 1858, Arthur Batut lifted cameras on kites in the late 1880s with a slow fuse to fire the shutter, and pigeons, masts and poles have all carried cameras since. The platform buys a viewpoint, not a different sensor.

5.1.6 Cameras beyond photography⧉

A camera can also return a measurement: the location of a manufacturing defect, a decoded label, or a robot's estimated motion. The appropriate test then concerns that output. An attractive image can be unsuitable for measurement if processing moves an edge or changes the relationship between pixel values and light.

Scientific cameras may use cooling, monochrome sensors, controlled illumination or selected spectral bands. Those choices address a specified task. A cooled astronomical camera, for example, reduces dark current during long integrations; it still has photon noise and requires calibration. The same distinction matters when a phone photographs a document: readable text and retained information take priority over an attractive rendering.

fig-cameras-beyond-photography
Figure 5.1.6. (Deferred, to be generated.) A strip of cameras that aren't for pictures, each labeled with its output rather than an image: machine-vision / industrial inspection (defect flag), barcode / QR reader (a decoded number), robot / drone (pose + depth via SLAM), automotive ADAS (detections), scientific / cooled-astronomy (a calibrated measurement), medical endoscope (a diagnostic view), security CCTV and AR/VR tracking. Same optics + sensor, different goal: accuracy, speed, and robustness over beauty.

The same hardware appears in several kinds of measurement:

Cooling illustrates the change in priorities. During a long astronomical integration, dark current accumulates alongside the wanted photoelectrons. Cooling reduces that unwanted rate, and a regulated sensor temperature makes matching calibration frames easier. Monochrome sensors with a filter wheel provide another choice: measure selected bands sequentially instead of spending spatial samples on a color mosaic. Scene motion, filter changes, and total observing time then become part of the measurement design.

5.1.7 A camera's other sensors⧉

A gyroscope measures angular velocity, and an accelerometer measures specific force. Together they form an inertial measurement unit (IMU). Their measurements can guide optical stabilization or initialize the alignment of video and burst frames. They need timing and geometric calibration relative to the camera; an IMU does not directly observe subject motion.

Position, heading and timestamps can help organize photographs. Ambient-light and flicker sensors can inform display or capture settings, and a range sensor can assist focus where image contrast is weak. Availability and use differ across devices. Check which measurements are actually stored if a later computation needs them; a feature in the camera does not imply access to its sensor stream.

fig-camera-sensor-suite
Figure 5.1.7. (Deferred, to be generated.) The sensors surrounding the image sensor in a modern camera/phone: the IMU (accelerometer + gyroscope) driving stabilization and alignment, GPS/GNSS and magnetometer for geotag and heading, barometer, ambient-light/flicker sensor, laser/ToF and lidar depth sensors, microphones, and housekeeping (temperature, Hall position, clock), with arrows marking which ones feed the computational pipeline versus the EXIF metadata.

The same motion measurement can support several operations. A gyroscope can guide a stabilizing lens element or sensor stage during capture, provide a starting guess for inter-frame registration, or constrain the camera-rotation component of a blur model. These uses ask for different information: a horizon indicator can tolerate an error that would be visible in pixel-accurate alignment. Timing matters as much as direction. If the motion samples are shifted relative to the exposure, even a well-calibrated angular-velocity measurement can predict the wrong image displacement. Subject motion must still be estimated from the scene or additional sensors.

Other measurements answer different questions. GNSS can record location; a magnetometer can contribute heading; a barometer measures pressure that can help estimate altitude. These do not all measure position directly, and their accuracy depends on conditions. A range sensor can assist autofocus, whereas a sensor that detects lighting flicker helps coordinate exposure with a varying source.

Temperature and position measurements can also be internal to the camera. Sensor temperature matters to dark-current calibration. Position feedback helps control a moving lens group or stabilization stage. A timestamp relates these measurements to the exposure. This is why the non-image sensors can affect how a picture is computed even when none of their data appears as a visible feature in the photograph.