11.6 Photo tourism🔗⧉

Type "Notre-Dame" or "Trevi Fountain" into a photo-sharing site and you get back tens of thousands of photographs taken by strangers who never coordinated with one another — every lens, every season, every time of day, every accidental tourist in the foreground. It looks like noise: a pile of redundant snapshots with no shared frame of reference. Snavely, Seitz and Szeliski's Photo Tourism (2006) made the startling claim that this pile is secretly a 3-D capture rig. Run structure-from-motion (SfM) over it and you recover, jointly and from the pixels alone, a sparse 3-D point cloud of the landmark and the camera pose — position and orientation — of every single photo, all in one coordinate frame. The collection stops being a heap of pictures and becomes a navigable place: click a window of the cathedral and the system glides you to the photograph that best shows it. The technique is foundational multi-view geometry; the data is the crowd's. Productised as Microsoft Photosynth, it became the public face of the idea that an unstructured internet photo collection is enough to reconstruct the world.

11.6.1 Structure-from-motion on internet collections🔗⧉

What makes the problem hard is precisely what makes the data free: the input is unstructured. A traditional 3-D scanner — a calibrated stereo rig, a turntable, a laser scanner — controls the cameras, the lighting, and the geometry, and knows its own parameters in advance. Photo Tourism is handed none of that. Its input is thousands of strangers' photos, each shot with an unknown camera at an unknown focal length, cropped and post-processed however the photographer pleased, under whatever weather and crowd happened to be there that day. Two photos of the same statue might be taken a decade apart, from opposite sides, one at noon and one at dusk, one with a wide lens and one zoomed in. The reconstruction has to discover the cameras' parameters and the scene and who was standing where, all at once, from this wild heterogeneity — and it has to do so while most candidate feature matches between any two photos are simply wrong.

The pipeline assembles tools from earlier in the book. First, detect and match features across image pairs: extract SIFT keypoints in every photo and match them by nearest neighbour in descriptor space (Sparse matching). SIFT's scale and rotation invariance is not a luxury here — it is the whole reason a feature on the rose window can be matched between a wide establishing shot and a tight zoom. Second, robustly estimate two-view geometry: for each promising pair of photos, fit the fundamental matrix $F$ — the $3\times3$ matrix encoding the epipolar geometry between two uncalibrated views (3D and depth) — inside a RANSAC loop that rejects the many outlier matches as it goes (Robustness: the ratio test and RANSAC). The ratio test prunes the ambiguous matches before fitting; RANSAC fits $F$ to a consensus of geometrically consistent inliers and discards the rest. A pair that yields a large, stable inlier set is a genuine overlap; a pair that does not is set aside. The output of this stage is a graph of photos connected by verified geometric relationships.

Third, and at the heart of it, reconstruct incrementally. Rather than solving for all cameras at once (which would be hopeless from a cold start), the system seeds the reconstruction from one well-matched pair, triangulates their shared inliers into initial 3-D points, and then adds cameras one at a time — each new photo brought in by matching its features to the points already reconstructed and solving for its pose. After every addition (and periodically over the whole set) it runs bundle adjustment: a large nonlinear least-squares optimisation that jointly refines all the 3-D point positions, all the camera poses, and all the camera intrinsics (focal length and distortion) to minimise the total reprojection error — the gap between where each 3-D point lands when projected into a photo and where its matched feature actually was observed (Triggs et al. 2000; 3D and depth). Bundle adjustment is the glue that keeps a growing reconstruction globally consistent; without it, small per-camera errors accumulate and the cloud drifts.

The result, drawn in the figure, is twofold. There is a sparse 3-D point cloud of the landmark — every triangulated feature a point in space — and there is a recovered camera frustum for every photo: a little pyramid marking where that photographer stood and which way they pointed, all in one shared coordinate frame. The snapshots have been placed back where they were taken. (Figure 11.6.1)

11.6.2 From reconstruction to experience🔗⧉

The geometry is the means; the experience is the point, and it is what made Photo Tourism a landmark beyond the SfM literature. Once every photo's pose is known, the collection becomes spatially browsable. You can fly a virtual camera through the point cloud, and for any viewpoint the system can snap to the real photograph whose recovered pose best matches where you are looking — so navigating the 3-D model is really navigating through other people's photographs of the place. Click a detail in one image and it understands, geometrically, which other photos show that same detail better, and offers to take you there. A flat gallery of unrelated snapshots becomes a coherent tour. Photosynth productised exactly this: upload a set of photos of a scene, get back a navigable synth.

The transitions are where the geometry earns its keep. Moving from one photo to another is not a hard cut or a dissolve; it is a 3-D-aware morph that uses the recovered cameras and points to interpolate a plausible in-between view — warping the source image toward the destination along the geometry rather than fading between two unrelated frames. The effect is that the collection feels like a continuous space you move smoothly through, not a slideshow you click between. This is the same "collection becomes an experience" idea that animates Photobios, where a personal set of portraits is morphed into a continuous time-lapse of an aging face. There the collection is one person across years and the navigable dimension is time; here the collection is a crowd at one place and the navigable dimension is space — but the move is identical: align a messy real collection to a shared frame, then let the data, not a synthetic model, supply the in-betweens.

11.6.3 Lineage🔗⧉

Photo Tourism opened a research line that scaled the same idea up and hardened it into standard infrastructure. Building Rome in a Day (Agarwal et al. 2009) took the machinery to city scale — reconstructing whole cities from hundreds of thousands of internet photos by parallelising the matching and reconstruction across a cluster and being clever about which of the quadratically-many image pairs are even worth attempting to match. The matching graph, not the optimisation, is the bottleneck at that scale, and much of the work is about pruning it.

The reconstruction machinery itself matured into a community standard: COLMAP (Schönberger & Frahm 2016) is the modern incremental-SfM (and multi-view-stereo) tool that most of the field now builds on, the spiritual successor to Snavely's original Bundler. When today's neural-rendering methods — NeRF and its descendants — need to know where each training photo's camera was, they almost always get those poses by running COLMAP first. So the data-driven, crowd-sourced reconstruction idea that Photo Tourism introduced did not just survive; it became the quiet pose-estimation substrate underneath a whole generation of 3-D and view-synthesis work (3D and depth).