Two hundred twenty feet below the western Pennsylvania countryside, 20 miles of roadway leads to the abandoned limestone quarry where Iron Mountain stores the paper archives of its corporate clients. The passage is barely wide enough for two cars, and the rough-hewn walls are a little too well lit. As you pass the locked entrances to rooms - caverns, actually - that encompass entire patent-application warehouses and film libraries, you feel like you're navigating through the brain of a slumbering giant. And there, in one of its farthest recesses,is where the beast stores the 11 million photographs that constitute the Bettmann Archive, perhaps the best-known collection of photos in the world.
Although the photos are kept in one room, their sheer quantity means that locating any one of them requires an elaborate ritual. Suppose you want to find an image of President Coolidge talking with Native Americans. First, researcher Robinya Roberts looks up "Coolidge" in a central card catalog that looks like it's been transplanted from your local library to the Bat Cave. Yellowed and worn, the 3-by-5 cards contain surprisingly little information: only a caption, a brief description, and a reference number. If a card seems promising - say, the one with a caption that reads "Calvin Coolidge Wearing Sioux Head Dress" - Roberts jots down the number and walks into another room, a cave 200 feet long, brightly lit and kept at a brisk 45 degrees. As if in one of Kafka's dreams, she walks along a line of filing cabinets that extends to the vanishing point, locates the appropriate drawer, and leafs through folders. Inside are original prints. If she finds one showing Coolidge conversing with a Native American, she looks on the back for another reference number, this one leading to an older set of cabinets deeper in the cave. There she finds the negative, wrapped in a fragile brown paper sleeve.
As it happens, Coolidge isn't talking to anyone in the headdress photo, so the search continues. Next, Roberts studies the ledgers that list Bettmann's subcollections. There, among the entries dated August 20, 1926, after "Wildwood NJ baby Parade" and before "Sioux Indian band," she sees the caption "Pres. Coolidge addresses Indians" written in a meticulous hand. A reference number leads back into the refrigerated vault where the negative lies in a decaying envelope, untouched for almost 80 years. A little dusting, scanning, and adjustment of the contrast, and a digitized photo made from the ancient negative will be on its way to the customer.
It's a fascinating business, but it's also a harbinger. As our hard drives fill up with thousands then tens and hundreds of thousands of digital snapshots, we're all going to face the same basic challenges as the Bettmann Archive. Of course, you won't haul 19 semitrailers full of your decaying family photos into cold storage, as the Bettmann did in 2001. But you can expect to go clicking through folder after folder of pixelated images, trying to find the one where Aunt Rose put on a silly inflatable life preserver in the summer of 1999.
A digital camera is an enticement to take more snaps than you can keep track of. Why not shoot a few more pictures of Aunt Rose to see if you can get one that's slightly more flattering to her - how to put it? - irregular profile. You don't even need to carry a camera; you can squeeze off a stream of shots with your cell phone and send them to your inbox. With the price of digital storage plummeting even as our time seems to become ever more valuable, it's cheaper to store the lot of them than to weed out the clunkers.
But having thousands of photos on a hard disk or DVD-ROM is the equivalent of throwing Bettmann's images into the air and letting them flutter to the ground. Our only hope is that the army of engineers laboring in labs around the world can come up with a better way.
The situation at the Bettmann Archive may be discouraging, but Corbis, a stock photo house that happens to be Bettmann's corporate owner, is groping toward a solution. For one thing, the 3.2 million images stored in Corbis' Seattle headquarters have been digitized - no more yellowing prints, cracking negatives, or dusty envelopes. For another, they're being annotated with keywords that make it possible for customers to log on to the company's Web site, sift through more than 62 terabytes of photo data, and pinpoint images that meet their needs.
This doesn't happen by magic. Inside the company's airy office, media cataloger Nick Fraser sits in front of two screens. On the right is a photo of a strawberry, perfectly lit and very red. On the left is an in-house software app that lets him browse through a list of keywords. Fraser's job is to tag images with appropriate terms so that when customers search using a keyword, they find photos that match. Fraser's palette offers 60,000 choices, arranged in a hierarchy; the top level is known internally as the "Top of the World." Under the category unaccountably called "Anatomy," he clicks on "Fruit." He adds "Red," and the photo is tagged.
A photo of a businessman sitting in a small chair facing two other chairs of increasing size is harder to tag. The picture is vague and evocative, the sort of image that might accompany a management essay in a business magazine. Deciding what words to use means figuring out what the photo might mean to others: Decisionmaking? Disappointment? Downsizing? There is no right answer. "You just have to look at it long enough to see what the metaphor is," Fraser says.
In computer parlance, these tags are metadata - information about information. But metadata as it relates to imagery is a slippery thing, and tagging is a craft, not a science. If, for example, Fraser doesn't recognize one of the figures in a cocktail party scene as Serena Williams and instead tags it "Nightlife," customers searching for photos of tennis stars won't find it, and it might as well not exist.
This process of manual metadata tagging, subjective and labor-intensive, may work for Corbis, but it's a lot to ask of the rest of us. Even when software developers try to make it easy, it's not easy enough. For instance, Adobe Photoshop Album offers a similar type of drag-and-drop labeling. Right now, you have to enter keywords manually; presumably someday you'll be able to upload the names of people, places, and events from your address book and calendar so at least you can drag and drop familiar names. Still, mere mortals don't have a 60,000-term online taxonomy or twin screens. More to the point, we don't want to hire Nick Fraser to do the job.
Thus, the metadata most of us attach to our photos is pretty pathetic. We can name them when we transfer them to a computer, but most people don't bother and end up with a hard disk full of photos with names like DSC00012.jpg and DSC00234.jpg. As the years go on, DSC00234.jpg will become an archaeological artifact that might as well be labeled Don't_Know_Don't_Care.jpg. If we're to have any hope of preserving our memories, we'll need to be more clever than that. Much more clever.
What do you do if you're too lazy - or overburdened or preoccupied - to tag your photos? Let a machine do it. Digital cameras already capture critical data points at the moment the shutter clicks. Most models record - in the image file itself - not only the date and time a photo was taken but also the focal length, the aperture setting, and whether the flash fired. These tidbits can provide clues about whether the photo was taken indoors or out, during the day or at night, focusing on something close up or far away. Scanty metadata, but potentially helpful.
But why limit the possibilities to what today's cameras can do? The image file format most cameras use includes fields for longitude and latitude, in anticipation of the day when global positioning systems are built in. That day could be soon. Cell phones already gather some positioning information, and by the end of 2005 all new cell phones in the US will be locatable to within 500 feet or so. Establish a Bluetooth wireless connection between phone and camera and the camera will know where it is. Web sites already exist that use GPS data to let you upload photos pegged to spots on maps, and a Stanford research project compares photos with shots of known locations, automatically annotating snaps with information about where they were taken.
Combine location data with a database that knows about places and public events and you can pinpoint pictures of Aunt Rose at the international volleyball semifinals. Link that with her personal calendar and you can differentiate between shots taken at the volleyball tournament and those shot at her 61st birthday beach party later the same day.
But there's even more metadata waiting to be gathered without lifting a finger. Presumably the most important pictures are the ones viewed, printed, or emailed most often. When it comes to searching for photos, that information can play the same role as the number of links to a Web page in Google's ranking algorithms.
"On my vacation I took 480 pictures, but there are only 40 or 50 good ones," says Lars Perkins, cofounder of Picasa. His company, which was recently acquired by Google, makes a photo management program that competes with Adobe's. "I can reduce the noise if my program can deduce which ones are important without making me tag them," he says. So Picasa notices which photos you print, which you email, and which you look at most often. When you do a keyword search, those pictures rank higher in the results. Future versions could evaluate overall image quality based on which images you enhance and crop, and even which you delete.
Microsoft wants to take this kind of context-based tagging a step further in its next-generation operating system, codenamed Longhorn. Unlike Windows XP, Longhorn can track information created by various applications. When you look up Aunt Rose in your contact manager, for example, the OS might offer access to any photographs you've emailed to her.
A Microsoft Research project called MyLifeBits provides clues about where this approach is headed. Jim Gemmell is one of the media-management experts working on MyLifeBits, a project that looks forward to a time when people record just about everything that happens to them via wearable videocams and other sensors. "When you return from a vacation, the system will make a travelog for you," Gemmell says. "It'll make maps of where you went and pick out nice, clear photos. Then you'll hit the button and they'll go straight to your blog, or your grandma." This world might not be as far off as it seems. "The Longhorn team wants to make sure something like MyLifeBits can be enabled by the next version of Windows," Gemmell says.
Of course, no matter how good software becomes at identifying photos based on what you do with them, that's just the beginning. The higher goal is to get a machine to do what people do without thinking about it: analyze what's in the picture.
Unfortunately, that turns out to be a technological conundrum. Computers are great at comparing patterns but terrible at recognizing abstraction. Consider scene recognition, a feature that's already available in products such as Adobe Photoshop Album, where it's labeled "Find by color similarity with this photo." In theory, your shot of the Grand Canyon, with its blue rectangle of sky on top and striated browns and ochers on the bottom, can serve as a model for finding other photos of the same place. But searching by color balance doesn't begin to address the difference between, say, Waikiki and St. Croix, much less between abstract categories like "beaches" or "fishing spots." It's helpful but limited, and we'll be lucky if the next generation of photo management tools can find "sunsets."
Even a pattern-recognition task as straightforward as identifying faces turns out to pose tortuous technical challenges. Although every face is a unique configuration of two eyes, a nose, and a mouth, all these features aren't always visible, and they can look different depending on expression, angle, and lighting. A human will have no problem recognizing the dark spot under Aunt Rose's nose as a shadow, but a computer won't know whether it's a mole, a scab, or a Hitler moustache.
Although the technology is improving, facial recognition recently failed in what has become the benchmark in the field: Computers tricked out with the latest software couldn't find the terrorist needle in the traveler haystack in a 2002 trial at Boston's Logan Airport. Consequently, this feature isn't likely to show up in a software menu anytime soon. "I'm not aware of anything available today that's even remotely close to what you'd want in a photo management system," says Michael Slater, a director of technical strategy at Adobe Systems.
Still, there's hope. An individual's network of family, friends, and associates may include only a few hundred faces; matching these to 10,000 photos is a far simpler task than comparing every airport traveler with photos of every suspected terrorist. So, even if the ability to identify faces isn't on Adobe's upgrade feature list, it could show up several revs down the line. After all, the consequences of error are lower than they would be at the airport: If your photo management program mistakes Aunt Rose for Cousin Joey, at least she won't be in for a cavity search.
If humans are so good at tagging photos and computers so bad at it, why not just give humans the job - but do it in a way that doesn't mire the process in drudgery?
Ben Shneiderman, a professor of computer science at the University of Maryland, put together a system that did just that. For the 2001 conference of the Association for Computing Machinery's Special Interest Group on Computer Human Interaction, Shneiderman set up kiosks where members could tag any of 3,300 photos taken over the past 20 years of meetings. Attendees dragged and dropped the names of the people they recognized. "Hundreds of users provided thousands of annotations," Shneiderman wrote in a postconference report.
Since the introduction of the Kodak Brownie more than 100 years ago, we've thought of photos as shiny paper rectangles stacked in shoe boxes or pasted to dusty albums, to be hauled out when we're feeling sentimental. But the connected world in which we live suggests a different approach. Private snaps are migrating to the Web as well as to closed social networks such as Flickr, where they potentially belong as much to their subjects as the person wielding the camera. If your family members could browse through your photos whenever they wanted, you wouldn't have to tag the photos featuring Aunt Rose, because she could do it herself. So could her children. Or crazy Uncle Fred, who has too much time on his hands.
And this may be the key to the future of photo management: Rather than locking pictures away, we'll make them public. Technology will imbue our images with a broader, deeper sense of shared memory. Our ways of finding photos will change - and with them, our ways of remembering.
Manual tagging
Most photo management programs let you tag pictures with searchable keywords. To minimize drudgery, some let you create a palette of phrases - names, locations, events - that can be dragged and dropped onto images.
On-location tagging
When you click the shutter, a digital camera captures not only a photo but also data that can help identify the image: date, time, whether the flash fired. Future cameras with built-in GPS receivers could record the location.
Data mining
Personal information stored in a your calendar, address book, and email app can provide clues for photo searches. For instance, software could look for the word "birthday" in your calendar and return shots taken on those dates.
Scene recognition
Scene recognition looks for similarities in color to identify photos taken in a particular setting, from a particular viewpoint - the sand-and-sky pattern of a beach, for instance.
Facial recognition
Current facial recognition software gets flummoxed by variations in lighting and angle. But eventually it might be accurate enough to sort a limited number of portraits, especially if they've been shot head-on under bright conditions.
Social networking
Computers have a hard time identifying faces, places, and events, but for people, it's a breeze. Sharing photos over a social network distributes the work of tagging by letting members enter details themselves.
David Weinberger (self@evident.com) is the author of The Cluetrain Manifesto and Small Pieces Loosely Joined.
credit: Richard Ballard
Feature:
>
Point. Shoot. Kiss It Good-Bye.
Plus:
>
