Morphing memories into 3-D

Dust off those photo albums, because new technology is breathing life into old vacation pictures.

Owing to the work of UW computer scientists, pictures uploaded to the photo-sharing Web site Flickr are taking on a whole new dimension. Their work has found success reconstructing the complex, three-dimensional geometry of objects such as the Cathedral Notre Dame using photos obtained from Flickr.

Led by Michael Goesele, a professor at Technische Universit[HTML_REMOVED]t Darmstadt in Germany and former UW postdoc, the group presented its work at the International Conference on Computer Vision in late October.

"Our work represents several years of work by a collaboration between the UW and Microsoft Research (MSR)," Goesele said.

The success in 3-D reconstruction did not develop apart from other lines of research.

"A pre-cursor to this project was the Photo Tourism work that Noah Snavely and Steve Seitz of UW developed with MSR's Rick Szeliski," said Brian Curless, a UW professor of computer science. "This work solved the problem of structuring the online photographs so that our multi-view stereo algorithm could then work its magic."

The new technique starts with a search query such as "Notre Dame Paris." This returns thousands of pictures, of which only the most similar are selected by the software. Blurry or obstructed photos are also filtered.

Next, photo tourism uses common features in the generally hundreds of photos remaining to build a spare 3-D model of the scene. As algorithms determine what vantage point the photo was taken from, depth is inferred and the model is filled out very accurately, approaching that of laser scanning, a more expensive, alternative method of 3-D reconstruction.

Integration of 3-D reconstruction into freely available applications may not be far off.

"One place you might see it is Google Earth or Microsoft Virtual Earth," said Steve Seitz, a collaborator and UW professor of computer science. "Microsoft Photosynth, available as a technology preview, is already using photo tourism. Precise 3-D reconstruction is a logical extension."

From a scientific perspective, the technology has transitioned from carefully determined laboratory photos to the unpredictable and general data Flickr members contribute.

Though the team has found success, working with uncontrolled data still provides challenges.

"One issue is coverage," Seitz said. "People don't naturally photograph scenes from every conceivable angle. This makes complete models more difficult to construct. Other limitations include reflective surfaces and dark areas."

The collaborative team has not shrunk from research obstacles. Work continues at the UW and Microsoft.

"This research opens many new doors," Goesele said. "It has led to a huge open space of problems. The collaboration is not finished."

Citing their latest success as a first step, the scientists expressed enthusiasm toward their ongoing work.

"We imagine that future algorithms will actually leverage the variability in online imageryfor even better results," Curless said."Moreover, we have only reconstructed a handful of sites, but the promise is there to scale up the algorithms to reconstruct entire cities."

[Reach reporter Brian Smoliak at news@thedaily.washington.edu.]