A better world view
Imagine moving about inside and outside St. Peter’s Square in Vatican City, soaring through a huge mosaic of digital images stitched together into a smooth tapestry that lets you dive into, rise above or circle around the environment.
Imagine that the gallery you’re exploring started with thousands of pictures posted online by people you’ll never meet and who don’t know each other.
You can do that already, thanks to fast-moving technology being developed in Seattle.
That experiment in massive digital photo-sharing is one example of dozens of cutting-edge hunts for smart applications that stitch together huge amounts of photos and give viewers the experience of being right in the middle of an environment.
The Seattle effort is called Photosynth, a technology project spearheaded by Microsoft but derived from efforts by researchers at the University of Washington.
The Photosynth Web site, at http:labs.live.com/photosynth, showcases several examples of how magical the end result is. The collections there use photos to produce six panoramas of interesting locations, including the South Korean Gyeongbokgung temple, the Grassi Lakes area of the Canadian Rockies and the Piazza San Marco in Venice.
The technology is so dazzling that the BBC used it create dazzling displays of six popular locales, including Ely Cathedral and Trafalgar Square (see the full BBC set at http://labs.live.com/ photosynth/bbc/default.htm).
Instead of gathering up the images and letting a computer try to navigate among them, the Photosynth tool produces a better, synthesized collective image of whatever the subject is, explained Steve Seitz, a UW researcher who helped develop the school’s prototype version of the product, called Photo Tourism.
Seitz worked on the project with UW doctoral student Noah Snavely and Richard Szeliski, an employee at Microsoft Research and affiliate professor at the UW.
In nearly every set of images, the pictures were found from free and public photo-sharing sites, such as Flickr.
The images are sorted and catalogued and the pictured object — a building or a large monument, for instance — is assigned a set of common reference points. All those points are plotted and are used to produce a 3-D “point cloud” of the location.
Using clever technology, each picture’s distance from those points is calculated and when the Photosynth program takes the viewer through or around the location, the various distances and angles are adjusted, to create a smooth and seamless display of the resulting images.
Seitz said the program is smart enough to adjust for different lighting in the source images, as well. “We use a feature-matching technique called SIFT, developed by David Lowe at the University of British Columbia, that handles very significant differences in lighting, shading, weather, scale, and so forth,” Seitz said.
Microsoft licensed the UW project and handed it over to the company’s Live Labs group. For now the group calls Photosynth a technology “preview,” meaning it’s not being sold but instead is offered to organizations, businesses and others to see how it can be developed.
It’s not clear what will happen next, said Adam Sheppard, the product group manager for Photosynth.
“We’re currently focused on continuing to improve the user experience, the time it takes to process the images and exploring community features for users to bring their photos together to create collective environments. We’ve been approached by many third parties and continue to evaluate commercial uses but do not have anything to announce as yet,” said Sheppard in an e-mail.
One ongoing concern is avoiding images that could be embarrassing or violate some individual’s right to privacy, Sheppard said. Microsoft noticed the protest against Google last month when it unveiled Street View, an application that gives viewers street-level, detailed pictures in five U.S. cities. (See it in action at http://maps.google.com/help/ maps/streetview).
Snavely, who continues work both at the UW and for Microsoft on the technology, said the difference between the two approaches is that Google’s project involves images it’s collected on its own and “owns,” he said. The Photosynth model relies on images produced and posted freely by users all across the planet, he noted.
He said one inviting facet of the technology is the possibility of using large sets of millions of images to produce larger-scale displays. “In effect, the option exists then that you give people a view of the world. And as that world or locations change, the people will update the photos, and so will the 3-D imagery used in Photosynth,” Snaveley said.
For now, Photosynth runs in either the Firefox or Internet Explorer browser. It currently is only available in Windows XP or Windows Vista. A high-end PC is recommended for good results.