BookNotesSlidesP-sets Syllabusdraft · v0.2.49.1
◀▲⌂▶
💬Comments welcome. To leave a note, select any text and click the note / highlight button that pops up — or open the panel with the tab at the top-right (‹). Notes are visible only inside our private review group.

38.8 Mm-wave, wifi⧉

We have spent this part stretching what counts as seeing by reading shadows, timing photons, deconvolving accidental apertures, but always with light. Now make the final move: change the wavelength so much that some obstacles to visible-light imaging become partly transmissive. Visible light has a wavelength under a micron and is stopped cold by a sheet of paper, let alone a wall. Radio transmission depends on frequency, material, thickness, and moisture. Some centimeter-wave systems can sense through drywall or wood; millimeter waves can penetrate clothing but many walls attenuate them strongly. Human bodies also produce useful reflected signals. So a radio transmitter on one side of a wall, and a receiver array on the same side, can bathe a hidden room in radio illumination and read back the echoes of the people inside. You can image through the wall with radiation outside the visible spectrum (Adib & Katabi 2013 (WiFi)).

38.8.1 Why radio walks through walls⧉

Transmission is governed by a material's frequency-dependent electromagnetic properties, as well as its thickness and structure. A material opaque in visible light can transmit useful radio energy, but wavelength alone does not decide transparency: visible light passes through ordinary glass, while metal can block radio. Water, reinforcing metal, and multilayer construction can strongly attenuate or scatter a radio probe. The body's water-rich tissues produce reflections that can be distinguished from the surrounding structure in favorable conditions. The result is a coarse, geometry-dependent echo pattern, not a transparent view of every room.

But radio buys that penetration at a steep price in resolution, and this is the tension the whole field fights. Diffraction ties the finest detail you can resolve to the wavelength and the aperture size. At centimeter wavelengths, with an antenna array you could fit in a room, the raw angular resolution is coarse, so you get blurry blobs, not crisp outlines. Millimeter-wave systems (the "mm-wave" of the title, including airport body scanners, 5G, and automotive radar) push to shorter wavelengths and larger effective apertures for sharper images, which is why the security-scanner picture is recognizably a body while a WiFi reflection is a smear. The art of RF imaging is extracting structure from a measurement that is, optically speaking, hopelessly blurry.

38.8.2 Time of flight, again: radar is NLOS with a longer wave⧉

The mechanism for turning radio echoes into geometry should feel familiar, because we just spent a chapter on it. Transmit a radio pulse (or a frequency sweep, which amounts to the same thing); measure how long the echo takes to return; the round-trip time gives the distance to whatever reflected it. This is radar, and it is the same time-of-flight reasoning as Chapter 38.6, Active non-line-of-sight, only with a radio carrier instead of a laser pulse and nanoseconds instead of picoseconds (both propagate at the speed of light in air; the usable signal bandwidth, rather than the carrier wavelength or target distance alone, sets range resolution). An array of antennas, each measuring the echo's timing and phase, lets you triangulate where the reflection came from, not just how far, which is the spatial-array analog of moving the laser spot across the wall. In effect, RF imaging is non-line-of-sight imaging where enough energy may penetrate the wall to illuminate a hidden region directly. Wall attenuation, refraction, reflections, and multipath still complicate reconstruction.

The motion of the body is a gift here, exactly as it was for the corner camera. A static room of furniture and a moving person produce very different echoes, and subtracting the static background (or filtering for the Doppler shift that only moving reflectors impose) isolates the human. The same approach of subtracting the static scene and keeping the change that powered Chapter 38.3, Motion and video magnification and Chapter 38.5, Corner camera reappears: motion is what lifts the body out of the clutter of wall and furniture reflections.

fig-rf-through-wall
Figure 38.8.1. Imaging through a wall with radio. Left: a transmitter and a receiver antenna array on one side of a wall emit radio waves (centimeter wavelength) that pass through the wall (low-density, low-conductivity, transparent to radio) and reflect off the water-rich human body inside. The round-trip time of flight of each echo gives range; the array's geometry gives direction, using radar triangulation, the radio counterpart of the laser time-of-flight in active NLOS. Center: the raw RF reflection is a coarse, blurry blob (diffraction limits resolution at long wavelength) with the static wall/furniture reflections differenced away so only the moving body survives. Right: the bridge to a usable result, picked up in the next figure. The figure's point: walls transmit, bodies reflect, and motion lifts the body out of the clutter. After Adib & Katabi 2013 (WiFi).

38.8.3 From a radio smear to a human skeleton: the learned map⧉

A blurry radio blob is not yet a useful picture of a person. The next step is to infer pose directly from the RF measurement. The blob is coarse, noisy, and nothing like the clean RGB image a camera would give. But it is statistically tied to the body's configuration: a given pose produces a characteristic pattern of reflections. So train a neural network to learn a mapping from RF reflections to human pose by feeding it radio measurements as input and have it output a 2-D or 3-D skeleton, the same joint locations a vision-based pose estimator produces.

The training setup uses cross-modal supervision. You cannot easily hand-label what a radio echo "should" decode to. But you can put an ordinary camera next to the radio system during data collection, run a standard vision pose estimator on the camera's images to get skeletons, and use those as the training labels for the RF network, so the camera provides supervision during training. At test time you throw the camera away and the network reads pose from radio alone, even through a wall or in total darkness, where the teaching camera would have been blind. This is RF-Pose (Zhao and colleagues, Zhao et al. 2018): radio in, skeleton out, supervised by vision it no longer needs. The earlier RF-Capture work (Adib and colleagues, Adib et al. 2015, a project I was part of) reconstructed a coarse human figure by stitching together reflections from different body parts as the person moved, demonstrating that enough of the body's geometry survives in the radio echo to recover a recognizable figure through a wall.

fig-rf-pose
Figure 38.8.2. Learning to read human pose from radio. Left: the coarse RF reflection of a person (from the previous figure), shown as a blurry, low-resolution heatmap, unrecognizable on its own. Center: a neural network maps the RF measurement to a 2-D/3-D skeleton (joint positions). The network is trained by cross-modal supervision: during data collection an ordinary camera films the same scene, a standard vision pose estimator extracts skeletons, and those serve as the labels that teach the RF network, with the camera providing supervision during training. Right: at test time the camera is discarded; the network reads pose from radio alone, working through a wall and in total darkness, where the teaching camera would see nothing. After Zhao et al. 2018 (RF-Pose); figure also reflects Adib et al. 2015 (RF-Capture).
Video 38.8.1. RF-Pose: through-wall human pose from radio (MIT CSAIL). The learned RF→skeleton map of Zhao et al. 2018 estimating human pose through a wall and in total darkness, supervised at training time by an ordinary camera it no longer needs at test time. (web edition)

38.8.4 What it sees, and what it costs us⧉

The capability list is uncanny. Research systems have demonstrated pose and gait estimation through suitable walls, and breathing and heart-rate estimation from the tiny periodic displacement of the chest, the radio echo of the same vital signs Chapter 38.3, Motion and video magnification pulled from video; it counts and localizes people in smoke or darkness; it powers contactless sleep and fall monitoring and through-wall search for first responders. It is, in the most literal sense, the completion of this part's project: revealing what is invisible here because it is behind a wall, sensed with light that is invisible because your eye has no receptor for it.

And it forces the part's quietest theme into the open: privacy. Every method in this part erodes the assumption that not being seen requires only blocking line of sight. The corner camera reads you from your shadow; passive periscopy reconstructs a room from a blank wall; RF sensing watches you through a solid wall, in the dark, with specialized radio hardware, antennas, and processing; these demonstrations are not equivalent to installing software on an arbitrary Wi-Fi router, and a learned model that turns the echo into your skeleton and your breathing. The same sensing principles motivate search-and-rescue applications and raise concerns about monitoring inside homes. Neither rubble penetration nor reliable pose recovery through arbitrary construction is guaranteed. There is no opting out of being a radio reflector. The technical lesson is that imaging is limited by what you can model and invert, not by what your eye can register, and some barriers that block visible light transmit a useful radio signal. The ethical consequence is straightforward: as we get better and better at revealing the invisible, the category of "private because unseen" keeps shrinking, and what protects people stops being physics and starts having to be choice.